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This  study  was  conducted  to  determine  whether  types  of  computer-related 

experience,  viewing  context  and  sequence  affect  the  ability  to  interpret  different 

functional  types  of  pictorial  sign  symbols  used  in  human-computer  interfaces.    Groups 

of  teacher  education  students  viewed  and  attempted  to  interpret  sets  of  pictorial  sign 

symbols  representing  common  word  processing  menu  commands,  constructed 

according  to  Knowlton's  taxonomy  of  functional  picture  types.    The  statistical  analysis 

used  a  four-way  ANOVA  based  ona3x2x3x3  factorial  design  with  repeated 

measures.    Factors  measured  included  computer  experience  type  (three  levels), 

context  (two  levels),  pictorial  sign  symbol  (PSS)  type  (three  levels),  and  sequence  of 

viewing  (three  levels). 


Main  effects  were  observed  for  computer  experience  type,  pictorial  sign 
symbol  type,  and  sequence.    A  significant  interaction  was  observed  between  the 
sequence  and  pictorial  sign  symbol  variables.   There  was  no  statistically  significant 
effect  due  to  context  as  operationalized  for  this  study. 

Analogical  PSSs  were  more  effectively  interpreted  by  subjects  with  all  types  of 
previous  experience,  and  logical  PSSs  were  the  least  well  interpreted  by  all  subjects. 
Realistic  PSSs  were  interpreted  more  accurately  during  the  second  and  third  sequence 
of  viewing  positions  than  during  the  initial  presentation.    Prior  experience  with  a 
graphic  user  interface  positively  affected  the  ability  to  interpret  PSSs. 

This  study  generated  evidence  that  may  be  useful  to  those  engaged  in 
designing  visual  messages  within  human-computer  interfaces  for  teacher  education 
students.   The  results  of  this  study  provide  support  for  the  systematic  use  of 
Knowlton's  taxonomy  in  the  design  and  evaluation  of  pictorial  sign  symbols,  and  the 
limitations  of  this  study  indicate  appropriate  directions  for  further  research.    This 
provides  empirical  information  that  should  be  useful  to  designers  of  computer 
systems,  designers  of  computer-based  instructional  systems,  designers  of  educational 
media,  and  educators  using  computers  as  tools  to  improve  learning. 


XI 


CHAPTER  1 
INTRODUCTION 


Designers  of  human-computer  interfaces  increasingly  use  pictorial  sign 
symbols  called  "icons"  to  communicate  messages  visually  to  computer  users.    Are 
there  differences  within  individuals  in  the  ability  to  interpret  these  symbols  due  to  the 
functional  type  of  sign  symbol  used?   Do  people  with  different  levels  of  computer- 
related  experience  differ  in  their  abilities  to  interpret  pictorial  sign  symbols 
representing  functions  of  a  computer  system?   Does  the  context  within  which  pictorial 
sign  symbols  are  viewed  affect  interpretation?   This  study  was  designed  to  find 
answers  to  these  questions. 

Statement  of  the  Problem 

Many  changes  have  occurred  over  the  last  century  in  the  types  of  educational 
media  used  in  the  schools.    Books,  globes,  and  maps  were  typical  visual  aids  used 
prior  to  1900  (Reiser,  1987).   The  development  of  film  in  the  early  part  of  the 
century  created  interest  in  and  resulted  in  research  into  visual  education  (Hoban  & 
Van  Ormer,  1950).   Investigations  into  the  educational  effects  of  television 
accompanied  its  widespread  distribution  during  the  1950s. 
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During  the  latter  third  of  the  century,  the  development  of  computers  and 

information  processing  technology  stimulated  further  investigations.   In  recent  years, 
investigations  in  educational  media  research  have  shifted  from  gross  media 
comparisons  of  the  effectiveness  of  technological  hardware  devices  to  issues  related  to 
instructional  messages  and  learner  characteristics  (Clark,  1989).    Some  researchers 
now  view  media  as  vehicles  for  the  delivery  of  instructional  messages  to  learners 
(Clark,  1983).   Educational  media  involve  the  symbol  systems  used,  the  messages 
conveyed,  the  learning  tasks  required,  and  the  characteristics  of  the  learner  (Salomon, 
1974).   Researchers  investigating  educational  media  should  look  at  the  symbol 
systems  and  the  conventions  of  visual  symbols  and  symbolic  codes  used  in  media 
(Clark  &  Salomon,  1986;  Levie,  1978). 

Learners  need  to  be  able  to  understand  the  symbol  systems  used  in  the  human- 
computer  interface.    Human-computer  interfaces  frequently  include  pictorial  symbols 
in  addition  to  textual  information.    Media  and  psychological  researchers  have 
produced  a  knowledge  base  and  provided  many  guidelines  for  the  creation  of 
communicative  visual  displays  (Fleming,  1989;  Fleming  &  Levie,  1978).    Media 
research  has  typically  involved  pictures  with  text.    Researchers  have  not  fully 
investigated  learning  from  pictures  alone,  which  remains  a  topic  of  continuing  interest 
(Gagne,  1986). 

The  growing  use  of  technology  has  also  increased  the  need  for  communication 
and  instruction  across  cultural  and  linguistic  barriers  (Frutiger,  1989;  Kolers,  1969; 
Mangan,  1978;  Modley,  1966;  Wildbur,  1989).    One  approach  to  this  problem  has 
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been  the  use  of  pictorial  sign  symbols  in  human-computer  interfaces  (Apple  Human 

Interface  Guidelines,  1987;  Lodding,  1983;  Nadin,  1988).   Terms  used  to  describe 
pictorial  sign  symbols  include  graphic  icons,  icons,  iconic  signs,  pictographs, 
pictographic  symbols,  and  picture  symbols.    For  the  sake  of  brevity,  pictorial  sign 
symbols  will  be  called   "PSSs"  throughout  this  paper. 

Designers  use  PSSs  to  make  the  human-computer  interface  more  easily 
understood,  learned,  and  used  by  novices  and  experts.   One  highly  successful 
approach  to  designing  the  human  computer  interface  has  been  the  use  of  a  "collective 
metaphor,"   such  as  the  office  desktop.   Systems  that  use  a  collective  metaphor 
represent  underlying  objects  or  functions  of  the  computer  by  incorporating  PSSs  into 
the  human-computer  interface.    The  Xerox  Star  office  automation  system  (Smith, 
Irby,  Kimball,  Verplank,  &  Harslem,  1982)  is  the  classic  example  of  this  type  of 
interface.    Unfortunately,  haphazard  and  inconsistent  imitations  based  on  marketing 
pressures  instead  of  the  users'  needs  in  the  specific  context  have  followed  (Gittens, 
1986;  Nadin,  1988). 

Investigators  have  often  studied  expert  and  technical  users.    As  access  to 
information  through  computer  systems  continues  to  expand,  the  focus  on  computer 
interfaces  will  move  away  from  technical  issues  toward  the  users  and  the  system 
interface  (Smithson  &  Hirschheim,  1990).    Many  issues  related  to  user  interfaces 
await  further  investigation.   Proponents  of  the  use  of  graphics  in  the  human-computer 
interface  recognize  that  PSSs  are  not  a  panacea;  there  are  several  problems  associated 
with  using  graphics  (Shneiderman,  1987).   Among  these  problems  are  (a)  the  visual 
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content  can  confuse  as  easily  as  it  can  clarify,  (b)  the  meanings  of  graphic  icons  may 

not  be  clear  to  all  users,  (c)  the  graphic  representation  when  understood,  may  mislead 
the  user,  (d)  graphic  icons  may  take  up  excessive  space,  and  (e)  manipulation  of 
graphic  objects  may  be  less  efficient  than  keyboard  input  for  some  situations. 

Computer  users  who  are  novices  in  the  use  of  a  system  actively  engage  in 
exploration  and  discovery  in  attempting  to  learn  how  to  use  a  software  application 
program  (Carroll  &  Mack,  1984).    In  this  situation,  the  human-computer  interface 
becomes  an  educational  medium  that  provides  cues  and  feedback  to  the  individual 
learner  seeking  to  achieve  unique  learning  goals.   Messages  to  and  from  the 
underlying  program,  and  the  user's  responses  to  the  program  create  an  interactive 
learning  dialogue.    In  the  graphical  user  interface,  PSSs  frequently  convey  messages 
between  the  system  and  the  user.    The  first  three  items  on  Shneiderman's  list  of 
problems  with  the  use  of  graphics  focus  on  the  problem  of  encoding  and  decoding 
messages.   Concerns  about  display  space  and  efficient  input  modes  are  secondary 
issues. 

Norberg  (1966)  noted  a  continuing  need  for  educational  media  research  that 
explores  the  potential  of  iconic  signs,  or  PSSs,  when  the  problem  involved  "helping 
the  learner  creatively  construct  a  response"  (p.  45),  such  as  that  described  by  Carroll 
and  Mack  (1984).   Until  recently,  most  computer-based  educational  materials  used 
digital  symbols  (written  text)  to  convey  messages  to  learners.    Media  researchers  have 
not  fully  explored  the  area  of  PSSs  dissociated  from  text.    Conversely,  human- 
computer  interface  design  has  involved  research  into  the  effects  of  PSSs  both  with  and 
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without  accompanying  text.   The  issues  of  encoding  and  decoding  the  messages 

delivered  through  PSSs  to  computer  users  are  primary  problems  in  the  use  of  graphics 
in  the  human-computer  interface  (Shneiderman,  1987). 

How  does  the  functional  type  of  picture  affect  the  ability  to  interpret  PSSs? 
How  does  prior  computer-related  experience  affect  the  ability  to  interpret  PSSs?   How 
does  the  viewing  context  affect  the  ability  to  interpret  PSSs?   Are  there  interactions 
between  the  functional  type  of  sign  symbol,  prior  computer- related  experience,  and 
context  that  affect  the  ability  of  the  learner  to  interpret  the  PSSs  used? 

Need  for  the  Study 

Pictures  and  pictograms  that  closely  resembled  the  designated  concepts 
preceded  and  eventually  developed  into  arbitrary  symbol  systems  (Frutiger,  1989; 
Knowlton,  1964).    Ironically,  the  growing  use  of  technology,  made  possible  by  the 
use  of  arbitrary  symbols  in  mathematics  and  science,  has  expanded  the  need  for 
universally  understood  non-arbitrary  sign  symbols  in  many  areas  of  communication 
and  education  (Frutiger,  1989).   PSSs  are  increasingly  used  as  a  language- free 
medium  for  communication  in  an  attempt  to  meet  this  need  (Easterby,  1970).   The 
increase  in  international  transportation  and  trade  during  the  latter  half  of  this  century 
has  expanded  the  move  for  standardization  in  the  use  of  PSSs  (Dreyfuss,  1966; 
Modley,  1966). 

The  incorporation  of  PSSs  or  icons  into  graphical  human-computer  interfaces 
has  occurred  primarily  during  the  last  decade  (Israelski  et  al.,  1989;  Lodding,  1983; 
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Smith  et  al.,  1982).   Most  human-computer  interfaces  had  been  text  based  before  the 

introduction  of  the  Xerox  Star  office  automation  system,  the  Apple  Lisa,  and  the 
Apple  Macintosh  (Nadin,  1988). 

If,  as  Olson  and  Bruner  (1974)  have  suggested,  intelligence  is  skill  in  the 
symbolic  systems  employed  in  the  media  used  in  instruction,  then  literacy  in  a 
medium  depends  on  the  ability  to  use  the  symbols  employed  in  that  medium 
effectively.   The  ability  to  use  computers  and  to  learn  through  computer-based 
instruction  has  been  highly  dependent  on  the  ability  to  use  written  language.    This  has 
led  to  concern  that  the  increased  use  of  technology  and  computer-based  instruction 
will  strengthen  rather  than  mitigate  the  literacy  bias  of  schools  (Clark  &  Salomon, 
1986). 

How  will  the  trend  toward  human-computer  interfaces  featuring  PSSs  affect 
learning  and  instruction?   Will  the  current  bias  toward  those  skilled  in  abstract  symbol 
manipulation  continue,  or  will  the  graphic  user  interface  ameliorate  the  advantages  of 
written  and  verbal  linguistic  skills?   How  will  the  type  of  graphic  objects  used  to 
represent  the  functions  and  commands  of  the  computer  system  affect  the  ability  of 
varied  users  to  understand  and  use  the  system  effectively?  This  study  was  conducted 
to  provide  data  relating  PSS  functional  types  to  the  ability  of  individuals  to  interpret 
intended  messages.    This  should  provide  empirical  information  useful  to  designers  of 
computer  systems  and  software,  designers  of  computer-based  instructional  systems, 
and  educators  using  computers  as  tools  to  improve  learning. 
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The  factors  involved  in  investigating  the  symbol  systems  used  in  the  human- 
computer  interface  require  a  multidisciplinary  approach.   Theory  and  research  from 
the  fields  of  computer  science,  instructional  design,  psychology,  human  factors, 
sociology,  communications,  graphic  design,  and  linguistics   contribute  to  an 
understanding  of  the  basic  issues  involved  in  the  design  and  evaluation  of  the  human- 
computer  interface  (Nadin,  1988;  Shneiderman,  1987).   Interface  designers  have  used 
semiotics,  the  theory  of  signs,  as  a  bridge  to  link  these  disparate  disciplines  (Nadin, 
1988). 

Educational  media  researchers  have  frequently  addressed  the  role  of  pictures 
with  text  instead  of  the  interpretation  of  PSSs  per  se,  because  pictures  typically  appear 
in  a  verbal  context  (Knowlton,  1966;  Levie,  1978).    Studies  of  pictures  used  in  direct 
instruction  have  resulted  in  guidelines  for  the  use  of  pictures  to  facilitate  the 
achievement  of  specific  learning  objectives  (Dwyer,  1978;  Kemp  &  Smellie,  1989). 
The  role  of  pictures  in  paired-associate  learning  has  also  been  widely  investigated. 
This  has  led  to  the  general  conclusion  that  pictures  are  superior  to  words,  and  words 
with  pictures  are  superior  to  words  alone  in  recognition  and  recall  tasks  (Pressley, 
1977).   Much  of  this  information  can  be  applied  to  the  design  of  the  human-computer 
interface. 

James  Q.  Knowlton  (1964,  1966)  created  a  taxonomy  of  pictures  based  on  the 
symbolic  sign  functions  of  pictures.   Within  this  taxonomy  there  are  three  types  of 
pictures;  realistic  pictures,  analogical  pictures,  and  logical  pictures.   There  has  been 
little  research  into  the  usefulness  of  this  taxonomy.    Researchers  in  this  area  have 
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reported  positive  results  (Alesandrini,  1989;  Hurt,  1987)  but  have  not  specifically 

addressed  the  use  of  PSSs  in  the  human-computer  interface.   Knowlton's  taxonomy 
was  applied  to  PSSs  as  used  in  the  human-computer  interface  in  this  study. 

Beginning  with  the  introduction  of  the  Apple  Macintosh  computer  in  1984, 
graphic  user  interfaces  featuring  PSSs,  commonly  called  icons,  have  become  widely 
available.    PSSs  are  often  thought  to  transcend  the  limitations  of  verbal  languages,  but 
in  actuality  are  not  universally  interpretable  (Kolers,  1969;  Mangan,  1978;  Modley, 
1966;  Sukaviriya  &  Moran,  1990).    Graphic  user  interface  designers  serving  users 
across  national  and  cultural  boundaries  need  to  consider  cultural  differences  in  the 
meanings  inferred  from  PSSs.   The  ideal  would  be  to  create  PSSs  that  do  not  rely  on 
verbal  labels,  and  to  incorporate  symbolic  conventions  that  are  globally  understood. 

Researchers  have  investigated  questions  regarding  the  design  of  instructional 
messages,  pictorial  symbols,  sign  systems,  and  the  human-computer  interface  for 
decades  (Collins,  1982;  Dreyfuss,  1966;  Easterby,  1970;  Fleming  &  Levie,  1978; 
Kolers,  1969;  Zwaga  &  Easterby,  1984).   These  researchers  have  generated  many 
guidelines  for  the  design  of  pictorial  symbols  (Apple  Human  Interface  Guidelines, 
1987;  Fleming  &  Levie,  1978,  Gittens,  1986;  International  Business  Machines 
Corporation,  1991;  Kemp  &  Smellie,  1989;  Lodding,  1983).    Designing  these 
symbols  still  appears  to  be  more  of  an  art  than  a  science  (Nadin,  1988).   Recent 
research  results  show  that  even  when  designers  follow  guidelines,  the  assumptions 
underlying  the  design  of  pictorial  symbols  are  not  always  supported  (Baecker,  Small, 
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&  Mander,  1991;  Blankenberger  &  Hahn,  1991).    Much  work  remains  to  be  done  to 

clarify  the  issues  involved  (Blankenberger  &  Hahn,  1991). 

A  person  must  have  attained  a  concept  before  a  symbol  that  refers  to  the 
concept  can  be  interpreted  correctly  (Knowlton,  1964,  1966;  Levie,  1978).   If  PSSs 
contribute  to  ease  of  learning,  then  what  types  of  computer-related  concepts  must  the 
learner  bring  to  the  human — computer  interface?   Results  of  recent  studies  indicate 
that  prior  experience  has  an  influence  on  the  ability  to  interpret  the  meanings  of  PSSs 
used  in  the  human-computer  interface  (Baecker,  Small,  &  Mander,  1991). 
Researchers  and  designers  have  recognized  that  different  levels  of  computer  users, 
from  naive  through  expert,  may  prefer  different  styles  of  dialogue  with  the  computer 
system  (Shneiderman,  1987;  Smithson  &  Hirschheim,  1990). 

The  meaning  of  a  symbol  may  vary  depending  on  its  context  (Mangan,  1978; 
Ossner,  1990;  Zwaga  &  Easterby,  1984).    Specificity  of  context  can  affect  the  ability 
to  interpret  a  pictorial  symbol  (Cahill,  1975;  Rogers,  1986).   Spatial  context  appears 
to  have  an  effect  on  interpretative  ability  even  when  general  conceptual  knowledge  is 
high,  in  some  contexts  (Cahill,  1975).   How  will  differing  contexts  affect  the 
interpretation  of  PSSs  with  varied  functional  types? 

This  study  was  designed  to  investigate  the  relationship  of  PSS  type  to  the 
ability  to  interpret  typical  messages  employed  within  human-computer  interfaces.    The 
results  of  previous  studies  have  not  supported  theoretical  assumptions  and  guidelines 
commonly  used  in  the  design  of  PSSs  for  human-computer  interfaces  (Blankenberger 
&  Hahn,  1991).    Some  investigations  into  the  improvement  of  the  interface  have 
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revealed  that  some  widely  used  PSSs  fail  to  communicate  clearly  (Baecker,  Small,  & 
Mander,  1991).   This  study  used  Knowlton's  taxonomy  of  pictures  (Knowlton,  1966) 
to  categorize  PSSs.    Subjects'  type  of  prior  computer  experience  was  used  as  a 
blocking  variable.   The  viewing  context  of  PSSs  was  manipulated  as  a  treatment 
condition. 

Definition  of  Terms 


The  following  terms  are  used  in  this  proposal  as  defined  below  and  as 
described  by  Knowlton  (1966).     These  terms  will  be  used  to  classify  PSSs  used  in  the 
study. 

A  realistic  picture  represents 
something  in  the  world  of  visual 
perception  and  refers  to  the  type  of 
object  portrayed.    For  example,  a 
graphic  representation  of  a  disk  drive 
may  be  used  to  refer  to  a  computer 
storage  device.   The  graphic 
representation  resembles  the  appearance 
of  a  computer  storage  device  (disk 
drive)  and  refers  to  the  type  of  object  portrayed.    A  PSS  used  to  convey  the  concept 
"Eject  Disk"  with  a  realistic  picture  could  represent  a  disk  in  a  disk  drive  and  indicate 


Figure  1-1  Example  of  a  Realistic  Picture 
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the  direction  of  movement.   See  Figure  1-1  for  an  example  of  a  realistic  sign 
symbolizing  the  command  to  eject  a  disk. 

An  analogical  picture  refers  to 
something  by  representing  something 
else  in  the  visual  world  as  an  example 
that  is  "like"  the  referent.   The  picture 
refers  to  something  other  than  itself  to 
communicate  this.    For  example,  a 
graphic  representation  of  a  filing  cabinet 

may  be  used  to  refer  to  a  computer 

Figure  1-2  Example  of  an  Analogical 
storage  device.   The  storage  of  data  in        Picture 

electronic  form  on  a  computer  storage  device  is  analogous  to  the  storage  of  data  on 

paper  in  an  office  filing  cabinet.   The  picture  of  the  filing  cabinet  does  not  look  like  a 

computer  storage  device,  but  it  is  analogous  to  the  computer  storage  device.   A  PSS 

used  to  convey  the  concept  "Eject  Disk"  with  an  analogical  picture  could  represent  a 

crumpled  piece  of  paper  being  thrown  into  a  trash  can.    See  Figure  1-2  for  an 

example  of  an  analogical  sign  symbolizing  the  command  to  eject  a  disk.   This 

particular  analogy  may  not  be  as  clear  as  the  filing  cabinet  analogy  and  may  have 

negative  connotations.   However,  it  is  very  similar  to  an  analogical  picture  commonly 

used  to  refer  to  the  "Eject  Disk"  function  in  a  commercially  available  microcomputer 

operating  system. 
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A  logical  picture  is  one  in  which 
the  elements  are  represented  in  an 
arbitrary  fashion.   The  graphic 
representation  refers  to  something  other 
than  itself  but  does  not  resemble  it. 
There  are  three  types  of  logical  pictures. 
These  include  pictures  whose  purpose  is 
to  represent  the  relationships  between 
elements,  pictures  whose  purpose  is  to 


Figure  1-3  Example  of  a  logical  Picture 


illustrate  or  represent  a  theory,  and  pictures  whose  referent  is  an  idea.    An  example 
of  the  first  type  of  logical  picture  is  a  road  map.    A  road  map  illustrates  the 
relationships  between  cities  and  towns,  but  the  small  circles,  squares,  or  stars  used  to 
indicate  cities  bear  no  relationship  to  the  actual  city  boundary  lines.   The  relationships 
of  the  cities  on  the  map,  and  the  contours  of  the  lines  representing  roads  connecting 
them,  do  resemble  the  actual  relationships.    An  example  of  the  second  type  of  logical 
picture  is  a  picture  of  the  structure  of  an  atom.    A  representation  of  an  atom,  with  a 
nucleus  with  electrons  circling  around  it,  does  not  resemble  an  actual  atom.   The 
intention  is  to  communicate  visually  a  theory  about  atoms.    An  example  of  the  third 
type  of  logical  picture,  which  refers  to  an  idea,  might  be  the  placement  of  a  sequence 
of  numbers  in  a  grid  with  equal  rows  and  columns  to  illustrate  the  concept  of  a 
number  squared.    A  graphic  representation  of  a  geometric  square  may  be  used  to 
refer  to  a  computer  storage  device.   The  "Eject  Disk"  concept  could  be  symbolized  by 
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placing  the  square  within  an  open-ended  box  and  indicating  the  direction  of 

movement.    See  Figure  1-3  for  an  example  of  a  logical  picture  symbolizing  the 
command  to  eject  a  disk.   This  could  be  considered  an  example  of  a  logical  picture 
signifying  relationships,  if  there  is  no  intention  to  represent  the  actual  physical 
appearance  of  a  computer  storage  device.    A  graphic  representation  of  the  directory 
"tree"  (file  structure  of  a  computer  storage  device)  would  be  an  example  of  a  PSS  that 
refers  to  an  idea.    Although  the  logical  arrangement  of  the  data  and  files  is 
isomorphic  with  the  tree  or  directory  structure,  in  reality  the  files  and  data  may  be 
non-contiguous,  scattered  in  clusters  over  the  physical  surface  of  the  disk. 

Limitations 

The  interpretation  of  results  from  this  study  is  subject  to  the  following 
limitations  and  assumptions.    Any  conclusions  are  limited  to  the  population 
represented  by  the  sample.   Generalizations  to  other  populations  can  only  be  made 
with  much  caution  and  subject  to  replication  of  results.   The  results  of  the  measures 
used  in  the  study  assume  that  subjects  answered  all  questions  independently,  honestly 
and  to  the  best  of  their  abilities. 

The  symbol  sets  used  in  this  study  were  constructed  to  avoid  possible  prior 
exposure  to  or  experience  with  these  specific  sets.    However,  some  correct 
interpretations  and  incorrect  interpretations  may  have  been  due  to  prior  experience 
with  highly  similar  PSSs  or  symbol  conventions.    Prior  computer  experience  was 
modeled  into  the  design  of  the  study  as  a  blocking  variable  to  minimize  this  threat  to 
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internal  validity.   The  results  of  this  study  are  limited  to  the  variables  as  defined  and 

manipulated.    Caution  should  be  used  in  generalizing  any  results  of  this  study  beyond 
the  specific  PSS  sets  used  in  this  study. 

Hypotheses 

This  study  was  conducted  to  find  answers  to  questions  related  to  the 
interpretation  of  PSSs  used  in  human-computer  interfaces.    Several  assumptions  about 
the  possible  relationships  among  the  variables  were  made  based  on  theory  and 
previous  research  findings.   Prior  experience  with  a  graphic  computer  interface  was 
assumed  to  have  a  positive  effect  on  the  ability  to  interpret  PSSs,  regardless  of  the 
sign  symbol  type.    Cues  provided  by  a  highly  specific  context  were  expected  to  have 
a  positive  effect  on  the  ability  to  interpret  PSSs,  regardless  of  the  sign  symbol  type. 

Therefore,  the  research  questions  were  stated  by  the  following  null  hypotheses: 

Hypothesis  1 .   There  will  be  no  significant  differences  within  subjects  in  the 
ability  to  interpret  different  types  of  PSSs. 

Hypothesis  2.    There  will  be  no  significant  differences  in  the  ability  to 
interpret  PSSs  between  subjects  viewing  PSSs  within  a  high  context  condition  and 
subjects  viewing  PSSs  within  a  low  context  condition. 

Hypothesis  3.    There  will  be  no  significant  differences  in  the  ability  to 
interpret  PSSs  between  subjects  with  different  types  of  computer  experience. 

Hypothesis  4.    There  will  be  no  significant  differences  in  the  ability  to 
interpret  PSSs  within  subjects  due  to  the  sequence  in  which  PSSs  are  viewed. 
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Hypothesis  5.    There  will  be  no  significant  differences  in  the  ability  to 

interpret  PSSs  due  to  interactions  among  the  functional  types  of  PSS  viewed,  the  type 
of  computer  experience,  the  viewing  context  conditions,  and  the  sequence  of  viewing. 

Summary 

Learners  using  computers  need  to  understand  the  symbol  systems  used  in  the 
human-computer  interface.   The  growing  need  for  communication  and  instruction 
across  cultural  and  linguistic  barriers  has  fostered  the  use  of  PSSs  in  human-computer 
interfaces  to  convey  information  to  users.    A  broad  knowledge  base  exists  for  the 
creation  of  communicative  visual  displays.   Much  is  known  about  learning  from 
pictures  with  text,  but  educational  media  researchers  have  not  fully  explored  learning 
from  pictures  alone.   The  trend  toward  the  use  of  PSSs  in  human-computer  interfaces 
has  generated  many  investigations,  but  theoretical  assumptions  have  not  always  been 
supported  by  the  results.     Comprehensive  prescriptive  guidelines  for  the  design  and 
implementation  of  PSSs  have  not  yet  been  established. 

The  following  chapter  includes  a  review  of  selected  theory  and  research 
concerning  what  is  known  and  what  remains  undetermined  about  the  interpretation 
and  comprehension  of  PSSs,  with  particular  emphasis  on  graphic  computer  interfaces. 
The  general  literature  review  is  followed  by  a  discussion  of  the  specific  studies  that 
led  to  the  questions  addressed  by  the  current  study. 


CHAPTER  2 
REVIEW  OF  RELATED  LITERATURE 


Research  into  the  use  of  pictorial  sign  symbols  (PSSs)  in  the  human-computer 
interface  has  lacked  a  unified  theoretical  basis.    Writers  in  the  educational  media 
literature  have  recycled  the  basic  principles  of  instructional  message  design  developed 
over  years  of  research  (Aspillaga,  1992;  Grabinger,  1989;  Hannafm  &  Hooper,  1989; 
Morrison,  Ross,  O'Dell,  Schultz,  &  Higginbotham-Wheat,  1989),  or  incorporated 
results  from  human  factors  research,  human-computer  interface  theories,  and  graphic 
arts  principles  (Lucas,  1991;  Reilly  &  Roach,  1986).   Although  much  of  this  material 
is  useful,  none  of  it  has  focused  on  the  symbol  systems  used  within  the  interface. 

Experts  have  proposed  taxonomies  for  PSSs  icons  used  in  the  human-computer 
interface  (Gittens,  1986;  Lodding,  1983),  but  have  not  systematically  investigated  the 
implications  for  application.   Similar  taxonomies,  using  categories  such  as 
"representational,"  "abstract,"  and  "arbitrary"  (Lodding,  1983);  "associative," 
"literal,"  and  "abstract"  (Gittens,  1986);  "concrete  object,"  "concrete  analogy,"  and 
"abstract  symbol"  (Rogers,  1986);  and  "iconic,"  "indexical,"  and  "symbolic"  (Nadin, 
1988)  have  been  used  to  classify  human-computer  interface  icons  in  similar  ways,  but 
there  has  been  no  consensus  of  terms  or  definitions.   This  lack  of  consistency  makes 
it  difficult  to  generalize  the  results  from  one  study  to  other  situations,  groups,  or 
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symbol  sets.    Nor  have  research  results  clearly  supported  the  theoretical  assumptions 

used  for  design  (Blankenberger  &  Hahn,  1991). 

Mihai  Nadin  (1988)  argued  for  the  use  of  semiotics,  the  theory  of  signs,  in  the 
design  of  the  human-computer  interface,  as  was  done  in  the  improvement  of  the 
original  Macintosh  interface.    Jervell  and  Olsen  (1985)  argued  that  semiotic  analysis 
of  icon  designs  indicated  that  these  PSSs  were  not  precise  in  meaning  and  that  context 
would  affect  interpretation.    The  continuing  trend  towards  the  use  of  PSSs  in  the 
human-computer  interface  suggests  that  it  might  be  useful  to  investigate  whether  some 
PSSs  are  more  effective  in  conveying  messages,  and  whether  there  are  any  systematic 
effects  of  context  on  interpretation. 

Knowlton's  (1966)  semiotic-based  taxonomy,  using  the  functional  categories  of 
"realistic,"  "analogical,"  and  "logical"  to  classify  pictures,  is  the  theoretical  basis  for 
the  primary  variables  manipulated  in  this  study.    Therefore,  this  review  of  related 
literature  will  begin  with  an  exposition  of  James  Q.  Knowlton's  semiotics-based 
taxonomy  of  pictures.   Selected  research  related  to  the  use  of  pictures  in  educational 
media  follows.   Human-computer  interface  designers  have  drawn  upon  human  factors 
research  involving  PSSs  used  in  traffic  control,  public  information,  and  instrument 
display  panels.    A  review  of  pertinent  results  in  this  area  precedes  selected  research 
on  human-computer  interaction.    Finally,  the  discussion  focuses  on  specific  studies 
related  to  the  issues  and  questions  addressed  in  this  study. 
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Knowlton's  Taxonomy 

The  use  of  signs  is  a  predominantly  human  trait.    No  other  animal  deals  with 
signs  with  equal  complexity  and  elaboration;  signs  are  basic  to  human  communication, 
society,  civilization,  and  science.    A  sign  is,  in  common  language,  something  that 
stands  for  or  refers  to  something  for  someone  (Morris,  1938). 

The  "something"  that  acts  as  a  sign  is  called  the  designatum,  object,  or  sign 
vehicle.   The  "something"  that  a  sign  stands  for  or  refers  to  is  the  referent.    The 
"someone"  to  whom  the  sign  vehicle  represents  the  designatum  is  the  interpreter;  the 
interpretant  is  the  effect  on  the  interpreter  (Knowlton,  1966;  Morris,  1938;  Nadin, 
1988).   Knowlton  classified  signs  produced  with  the  intent  to  communicate  as  symbols 
(Knowlton,  1964,  1966).   Pictorial  sign  symbols  are  pictures,  graphic  visual  displays, 
or  visual-iconic  representations  produced  with  the  intent  to  communicate  by  acting  as 
sign  vehicles. 

For  example,  a  picture  of  an  electronic  hand  calculator  within  the  visual 
display  of  a  human-computer  interface  may  refer  the  user  to  a  program  that  emulates 
a  hand  calculator  or  to  a  mathematical  calculation  function;  the  pictorial  representation 
chosen  will  influence  the  user's  interpretation  (Nadin,  1988).    A  program  that 
emulates  the  capacity  of  a  simple  hand  calculator  may  visually  display  a 
representation  of  the  calculator  on  the  screen.    The  visual  display  of  a  calculator  is 
the  sign  vehicle.    The  underlying  function  of  the  computer  system  is  the  object  or 
designatum.   The  interpretant,  or  effect  produced  on  the  interpreter  will  depend  on 
the  prior  knowledge  and  experience  of  the  interpreter,  as  well  as  the  particular  context 
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within  which  it  appears.   Within  a  consistent  desktop  office  environment  metaphor, 

this  symbol  might  be  interpreted  as  a  function  that  emulates  a  hand  calculator.   Within 
a  word  processing  or  spreadsheet  application,  a  calculator  icon  might  be  interpreted  as 
a  mathematical  equation  editor,  a  recalculation  function  button,  or  a  feature  providing 
access  to  built-in  math  and  financial  functions.    Whether  the  interpreter  correctly 
infers  the  referent  of  the  PSS  may  depend  on  interaction  between  the  prior  knowledge 
and  experience  of  the  interpreter  and  the  context  within  which  the  sign  is  perceived. 

The  "something"  that  a  sign  refers  to  does  not  have  to  be  a  tangible  or  even  a 
real  object— the  referent  or  designatum  is  not  a  thing,  but  a  category  or  class  of 
objects  (Knowlton,  1966;  Morris,  1938).    Sign  categories  that  refer  to  referent 
categories  are  defined  by  their  attributes,  the  relevant  cues  that  allow  discrimination 
among  categories  (Knowlton,  1966).    Knowlton  identified  two  types  of  attributes. 
Formal  attributes  refer  to  the  perceptual  qualities  of  a  sign  or  an  object,  such  as  color, 
size,  and  weight.   Criterial  attributes  refer  to  those  features  used  as  keys  for  sorting 
and  discriminating  between  classes.    Criterial  attributes  depend  upon  the  context 
within  which  a  sign  is  perceived.    Formal  attributes  such  as  size  and  shape  would  be 
non-criterial  for  separating  limes  and  lemons.    Color  would  be  a  criterial  attribute  for 
this  task. 

Signs  can  be  categorized  along  the  continuum  of  generality  and  specificity  as 
equivalence  and  identity  categories  (Knowlton,  1966).    Equivalence  categories  consist 
of  broad  or  generic  terms  such  as  "animal,"  "man,"  "statue"  or  "house."   The 
members  of  this  type  of  class  of  items  are  perceived  as  separate  entities  that  share  a 
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common  attribute  or  set  of  attributes.    An  equivalence  category  would  be  "statues"  or 

"houses."    Elements  of  this  category  would  be  seen  as  different  entities,  sharing  a 
common  definition.   The  identity  category  consists  of  items  such  as  "The  Statue  of 
Liberty"  or  "The  White  House."   Elements  of  this  category  would  be  considered  as 
different  aspects  of  the  same  entity. 

Pictorial  sign  symbols  used  in  a  human-computer  interface  are  often  classified 
according  to  their  relative  position  along  a  continuum  of  concreteness  and  abstractness 
(Nadin,  1988,  p.  67).    Concreteness  used  in  this  sense  indicates  a  high  level  of 
iconicity.    Concrete  images  bear  a  high  visual  resemblance  to  the  thing  or  object 
depicted,  while  abstract  images  have  less  detail.   Concrete  images  tend  to  be  iconic, 
and  their  interpretation  is  based  on  recognition.    Abstract  images  tend  to  be  more 
symbolic,  and  a  greater  amount  of  convention  is  involved  in  their  interpretation. 

Each  of  the  items  signifying  "The  Statue  of  Liberty,"  for  instance,  can  be 
considered  media  for  transmitting  a  concept,  or  shared  meaning.   The  medium, 
whether  photograph,  drawing,  silhouette,  spoken,  or  written  word,  is  the  sign  vehicle 
that  delivers  the  message  to  the  recipient.   The  shared  meaning  (conventional  response 
and  interpretation)  is  the  sign,  or  sign  symbol,  in  Knowlton's  taxonomy.   We  do  not 
usually  make  a  verbal  distinction  between  the  medium  and  the  message  by 
differentiating  between  the  sign  vehicle  and  the  sign  symbol  (Knowlton,  1966).    For 
this  study,  the  nature  of  the  sign  vehicles  used  is  indicated  by  use  of  the  term  pictorial 
sign  symbol. 
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An  iconic  sign  is  one  that  resembles  that  for  which  it  stands  (Knowlton,  1966; 

Nadin,  1988).   A  digital  sign  bears  no  resemblance  to  its  referent,  and  is  arbitrary, 
determined  by  convention.   Examples  of  digital  signs  include  words,  numbers,  sign 
languages,  and  codes  (Knowlton,  1966).    A  pictorial  representation  of  a  calculator 
would  be  an  iconic  sign.    Representations  of  the  symbols  used  to  represent 
mathematical  calculations  for  addition,  subtraction,  multiplication,  and  division,  or  the 
word  "calculator"  would  be  digital  signs. 

Knowlton  noted  that  sign  vehicles  high  in  iconicity  will  generally  depict 
identity  categories,  while  single  words  typically  depict  equivalence  categories.    In 
order  to  broaden  the  message  conveyed  and  interpreted  by  a  picture,  the  picture 
would  be  stripped  of  detail  until  the  criterial  attributes  alone  remained.    Examples  of 
the  continuum  of  concreteness  to  abstraction  are  provided  by  the  evolution  of 
characters  in  a  pictographic/ideographic  writing  system  such  as  Chinese  (Knowlton, 
1966),  the  evolution  of  a  pictorial  representation  of  a  calculator  (Nadin,  1988,  p.  71), 
and  the  evolution  of  the  phonetic  character  "A"  (Frutiger,  1989,  p.  145).   Pictures  that 
are  concrete,  realistic,  and  detailed  representations  tend  to  be  interpreted  as  individual 
entities  within  the  identity  category,  while  pictures  stripped  of  detail  or  local  features 
tend  to  be  interpreted  within  the  equivalency  category. 

The  aspects  of  Knowlton's  taxonomy  described  above  are  common  concepts  in 
the  field  of  semiotics.  There  are  two  aspects  of  Knowlton's  taxonomy  that  do  offer  a 
unique  scheme  for  categorization.  First  is  the  specification  of  the  varied  functions  of 
pictures  as  visual-iconic  signs  into  three  different  categories.  The  second  aspect  is  that 
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this  classification  of  pictures  by  their  functional  meanings  is  largely  independent  of 

the  physical  attributes  of  the  pictures. 

Visual  classification  schemes  may  discuss  physical  characteristics  such  as  size, 
shape,  and  color  (e.g.,  Gittens,  1986).   If  the  issue  of  interest  is  the  interpretation  of 
the  meanings  of  symbols,  however,  a  taxonomy  based  on  physical  attributes  is 
inadequate.    Knowlton  classified  pictures  as  visual-iconic  signs  according  to  the 
function  of  the  picture  as  realistic,  analogical,  or  logical,  independently  of  the 
physical  attributes  of  the  symbol.   To  classify  a  PSS  within  Knowlton's  taxonomy, 
one  must  consider  the  verbal  context  within  which  the  PSS  appears. 

The  parts  of  a  visual-iconic  sign  include  the  elements,  their  pattern  of 
arrangement,  and  their  order  of  connection  (Knowlton,  1966).   For  example,  in  a 
diagram  of  a  circuit  board,  the  elements  are  the  memory  chips,  transistors,  resistors, 
capacitors,  and  other  items.   The  pattern  of  arrangement  would  be  the  spatial  layout 
of  these  items  in  the  diagram.   The  order  of  connection  would  be  the  sequence  in 
which  these  elements  are  connected.    Knowlton  considered  a  visual  representation  to 
be  a  visual-iconic  sign  if  the  representation  of  at  least  one  of  the  three  categories  of 
parts  was  nonarbitrary.   The  types  of  representation  are  iconic,  analogical,  and 
arbitrary.   Pictures  with  iconic  representation  of  elements  are  categorized  as  realistic 
pictures.    Pictures  with  iconic  elements  that  use  real  world  objects  to  refer  to  other 
objects,  actions,  or  concepts  are  categorized  as  analogical  pictures.    The  picture 
element  may  be  a  representation  of  a  concrete  object,  but  it  is  used  to  refer  to 
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something  else.    Pictures  with  arbitrary  representation  of  elements  are  categorized  as 

logical  pictures. 

Knowlton's  taxonomy  is  based  on  the  function  of  the  PSS  in  communicating  its 
meaning,  instead  of  its  physical  characteristics  alone.    An  iconic  representation  of  a 
hand  calculator  that  refers  to  a  computer  program  that  faithfully  replicates  the 
appearance  and  functions  of  a  hand  calculator  could  be  considered  a  realistic  picture. 
It  conveys  its  meaning  by  directly  resembling  its  referent.    An  iconic  representation  of 
a  hand  calculator  that  refers  to  a  spreadsheet  function  such  as  "recalculate"  would 
probably  be  considered  an  analogical  picture.   Although  it  is  an  iconic  representation 
of  a  concrete  object,  it  conveys  meaning  through  analogy.   The  analogical  picture 
uses  the  real  world  object  (calculator)  to  refer  to  something  other  than  itself  (the 
process  of  recalculating  the  values  of  all  the  cells  within  the  program  data)  to 
communicate  this.    An  iconic  representation  of  a  mathematical  square  root  symbol 
that  refers  to  a  group  of  built-in  functions  would  be  considered  a  logical  picture, 
because  the  symbol  is  arbitrary  and  does  not  "resemble"  a  square  root. 

The  critical  difference  in  Knowlton's  taxonomy  is  the  classification  of  PSSs  by 
their  functions  instead  of  solely  by  their  physical  attributes.    Knowlton  (1966)  noted 
an  illustration  of  two  lumberjacks  working  together  to  support  or  move  a  felled  tree. 
The  picture  of  the  lumberjacks  and  the  trees  is  meant  to  depict  through  analogy  how 
muscles  are  attached  to  the  bones  of  the  skeleton  and  work  together  to  move  in 
different  directions.    Although  the  physical  attributes  of  the  picture  might  classify  it  as 
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a  realistic  depiction  of  lumberjacks,  the  function  of  the  sign  vehicle  in  this  context  is 

such  that  it  is  categorized  as  an  analogical  picture. 

Another  detail  in  Knowlton's  taxonomy  that  other  schema  lack  is  the 
identification  of  elements,  pattern  of  arrangement,  and  order  of  connection  in 
determining  to  which  category  a  picture  belongs.   Assignment  of  a  picture  to  a 
realistic,  analogical,  or  logical  sign  symbol  category  depends  on  the  functional 
representation  of  the  elements  of  the  picture.    The  pattern  of  arrangement  and  the 
order  of  connection  of  the  elements  may  or  may  not  be  congruent  with  the  function  of 
the  elements.    For  example,  arrows  are  often  used  within  PSSs.    An  arrow  may  be 
used  in  a  sign  symbol  as  an  element  or  as  part  of  the  pattern  of  arrangement  or  order 
of  connection.   An  arrow  that  is  itself  a  PSS  intended  to  communicate  the  message 
"Advance  to  next  screen"  is  an  element,  and  is  an  example  of  a  logical  picture.    An 
arrow  that  is  part  of  a  PSS  portraying  a  piece  of  paper  being  thrown  into  a  trash  can 
is  an  arbitrary  portrayal  of  the  pattern  of  arrangement  (the  paper  is  outside  the  trash 
can,  traveling  on  a  trajectory  that  will  end  inside  the  trash);  the  symbol  itself  is  an 
analogical  picture  for  the  "delete"  function.   An  arrow  that  is  part  of  a  PSS  consisting 
of  a  rectangular  area,  a  document,  and  an  arrow  pointing  from  the  document  to  the 
rectangular  area  would  represent  the  order  of  connection,  or  the  flow  of  data  between 
the  elements.    A  bi-directional  arrow  in  such  a  display  might  indicate  a  bi-directional 
connection  between  the  document  and  the  rectangular  block. 

Semiotics  theory  developed  by  C.S.  Peirce  identifies  three  kinds  of  signs: 
iconic,  indexical,  and  symbolic  (Berger,  1989;  Nadin,  1988).     An  iconic  sign 
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signifies  by  resemblance  to  the  thing  for  which  it  stands.    This  is  consistent  with 

Knowlton's  taxonomy.   An  indexical  sign  is  a  sign  that  signifies  through  a  logical 
connection  to  the  referent  (Berger,  1989)  or  through  a  representation  of  something 
causally  influenced  by  the  referent  (Nadin,  1988).    A  symbolic  sign  signifies  through 
convention  and  must  be  learned  (Berger,  1989;  Nadin,  1988).   In  the  example  of  the 
calculator,  an  indexical  sign  symbol  could  be  a  picture  of  a  paper  printout  ribbon, 
showing  the  results  of  the  calculations.    A  symbolic  sign  symbol  could  be  a  picture  of 
the  conventional  mathematical  symbols  used  to  represent  addition,  subtraction, 
multiplication,  and  other  functions  performed  by  a  calculator,  according  to  Nadin. 

Nadin  presented  a  table  of  varied  PSSs  representing  a  hand  calculator  with 
iconic,  indexical,  and  symbolic  representations  on  one  axis  and  a  continuum  of 
concreteness-abstractness  on  the  other  axis  (Nadin,  1988,  p.  67).     Further  analysis 
reveals  that  the  example  of  an  indexical  sign  provided  by  Nadin  does  not  differentiate 
between  the  iconic  and  indexical  representations,  but  represents  a  calculator  by 
synecdoche,  in  much  the  same  fashion  that  a  handset  represents  a  telephone. 

Realistic,  analogical,  and  logical  pictures  can  be  used  effectively  in  instruction, 
but  there  has  been  no  systematic  investigation  using  Knowlton's  taxonomy 
(Alesandrini,  1989).    Alesandrini  discussed  the  use  of  learner-generated  realistic, 
analogical,  and  logical  computer  graphics  to  enhance  learning  of  science  topics,  but 
did  not  address  the  design  of  the  human-computer  interface.    Hurt  (1987)  has  shown 
that  the  type  of  information  learned  and  recalled  from  text  can  be  affected  by  the 
functional  type  of  picture  accompanying  the  text.   This  suggests  that  an  analysis  of  the 
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type  of  PSSs  used  in  the  human-computer  interface  could  be  used  to  align  instruction 

with  the  type  of  sign  presented  by  the  computer.   If  the  print  function  is  represented 

by  a  realistic  PSS  of  a  printer,  then  instruction  should  be  most  effective  if  it 

emphasizes  that  this  symbol  is  used  to  send  information  to  the  printer  to  be  printed 

out  on  paper.   If  an  editing  function  is  represented  by  analogical  PSSs  of  scissors  and 

a  paste  jar,  then  instruction  should  be  most  effective  if  it  emphasizes  that  moving  and 

relocating  information  within  the  computer  file  is  analogous  to  cutting  out  a  part  of  a 

printed  text  document  with  scissors  and  pasting  it  in  a  different  location  within  the 

overall  document.    If  the  online  help  function  is  represented  by  a  logical  PSS  of  a 

question  mark,  then  instruction  should  be  most  effective  if  it  emphasizes  the  link 

between  the  question  mark,  an  arbitrary  symbol  used  to  represent  inquiry, 

inquisitiveness,  or  confusion,  and  the  process  of  seeking  information,  directions,  or 

help  by  asking  questions.   These  assumptions  could  be  tested  empirically.   This 

study's  purpose  is  to  investigate  the  effects  of  functional  types  on  the  interpretation  of 

PSS,  not  to  confirm  the  generalizability  of  previous  research  to  the  task  of  learning  to 

use  a  computer. 

To  summarize,  the  classification  of  a  picture  according  to  Knowlton's 

taxonomy  depends  on  the  intended  function  of  the  sign  as  a  realistic,  analogical,  or 

logical  sign  symbol,  and  not  solely  on  the  physical  attributes  of  the  picture 

(Knowlton,  1966).   The  classification  also  depends  on  the  signification  of  the  elements 

as  iconic,  analogical,  or  arbitrary,  and  not  their  patterns  of  arrangement  or  order  of 

connection.    This  classification  was  chosen  for  this  study  due  to  the  potential  to 
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describe,  explain,  and  predict  more  fully  the  interpretation  of  PSSs  used  in  human- 
computer  interfaces.    The  commonly  found  bipolar  schema  of  concrete-abstract  and 
realistic-symbolic  visual  iconic  signs  are  compatible  with,  but  do  not  describe  the 
functions  of  PSSs  as  comprehensively  as  Knowlton's  taxonomy.    Knowlton's 
taxonomy  will  be  used  later  in  this  review  to  analyze  specific  research  findings  related 
to  the  use  of  PSSs  in  the  human-computer  interface.    At  this  point  the  review  will 
turn  to  a  consideration  of  the  educational  media  literature  as  it  relates  to  the  study 
topic. 

Educational  Media 

Interest  in  the  use  of  PSSs  has  been  evident  in  the  educational  media  and 
instructional  design  field  for  several  decades  (Knowlton,  1964,  1966;  Levie,  1987; 
Norberg,  1966).    Overall,  questions  have  shifted  from  the  study  of  stimuli  within  the 
behaviorist  paradigm  to  the  exploration  of  learner  characteristics  and  internal 
cognitive  processes  (Clark  &  Salomon,  1986;  Olson  &  Bruner,  1974;  Salomon, 
1974). 

Many  of  the  principles  derived  from  psychological  research  and  applied  to 
educational  media  (Fleming  &  Levie,  1978)  also  apply  to  the  design  of  the  human- 
computer  interface  and  PSSs  used  in  the  interface.    For  example,  Fleming  and  Levie 
made  the  following  generalizations  from  basic  research  in  the  behavioral  sciences: 

1.  Material  that  is  meaningful  to  the  learner  is  acquired  and  retained  more 
easily. 
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2.  Concreteness  facilitates  learning  and  remembering. 

3.  Pictures  are  more  easily  remembered  than  words,  and  concrete  words  are 
more  easily  remembered  than  abstract  words. 

4.  Combined  text  and  graphics  should  be  congruent  or  redundant,  so  that  the 
learner  receives  the  same  message  regardless  of  the  channel  of  communication 
attended  to. 

5.  Messages  to  the  novice  within  a  domain  should  begin  with  the  more 
concrete  and  move  to  the  more  abstract  as  the  learner  increases  knowledge. 

6.  Cues  that  are  familiar  or  direct  attention  to  relationships  can  facilitate 
learning. 

7.  Discriminations  are  learned  most  readily  by  progressing  from  maximal 
differences  to  finer  differences. 

These  principles  or  generalizations,  based  on  a  variety  of  studies,  are  among 
those  applicable  to  the  analysis  of  the  human-computer  interface.  They  are  restated 
below  as  guidelines  for  designing  PSSs  for  human-computer  interactions. 

1.  Use  images  that  are  familiar  to  and  meaningful  to  the  user/learner. 

2.  Use  concrete  images  rather  than  abstract  images. 

3.  Use  pictures  rather  than  words  when  possible,  and  use  concrete  words 
rather  than  abstract  words. 

4.  When  combining  text  with  a  graphic,  both  text  and  graphic  should  convey 
the  same  message. 
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5.  Pictorial  sign  symbol  sets  for  novices  should  use  more  concrete  images; 

pictorial  sign  symbol  sets  for  experts  may  use  more  abstract  symbols  and  images. 

6.  Include  familiar  or  directional  cues  to  facilitate  interpretation  of  pictorial 
sign  symbols. 

7.  Menu  and  command  options  should  use  images  that  are  maximally  different. 
Simply  adapting  guidelines  established  with  print-based  media  may  not  result 

in  the  same  effect  on  learning  using  computer  screens  (Hannafin  &  Hooper,  1989). 
Without  systematic  research  directed  towards  screen  design  issues,  instructional 
designers  need  to  be  cautious  in  the  application  of  intuition,  common  sense,  or  rules 
developed  with  print  media  to  a  more  interactive  interface.   The  results  of  recent 
research  have  supported  and  contradicted  guidelines  and  assumptions.    Specific  issues 
will  be  noted  later  within  this  review. 

The  educational  media  literature  regarding  computer  screen  design  typically 
addresses  recommendations  regarding  textual  issues  (Grabinger,  1989;  Morrison  et 
al.,  1989).   Recommendations  include  admonitions  to  use  lower  case  text  and  ragged 
right  margins  for  ease  of  readability,  use  appropriate  amounts  of  white  space  between 
lines,  and  provide  a  visual  structure  for  the  learner  (Aspillaga,  1992).   Though 
previous  research  involving  typographic  features  has  informed  suggested  guidelines 
for  computer-based  instruction,  the  results  do  not  always  meet  expectations.    For 
instance,  when  provided  sufficient  contextual  support,  some  subjects  prefer  a 
computer  screen  that  has  more,  rather  than  fewer  words,  as  suggested  by  previous 
research  (Morrison  et  al.,  1989). 
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Recently,  attention  has  moved  toward  the  visual  design  of  the  human-computer 

interface  (Lucas,  1991;  Reilly  &  Roach,  1986)  and  toward  the  use  of  visuals  with  or 
without  computer  displays  (Frascara,  1984).   Writers  in  this  field  caution  against 
adherence  to  simplified  rules  (Hannafin  &  Hooper,  1989).   Greater  attention  is  needed 
for  the  relationship  between  task  attributes  and  the  capabilities  of  learners  (Hannafin 
&  Hooper,  1989).    There  are  many  topics  to  be  explored  in  this  area,  including  icons 
or  PSSs;  future  research  should  focus  on  smaller  subtasks  rather  than  overall  learning 
(Grabinger,  1989). 

Inquiries  into  the  human-computer  interface  may  use  information  produced 
through  studies  involving  familiar  media.    For  instance,  Beck  (1991)  relates  the 
results   of  studies  involving  text  and  arrow  cues  in  pictorial  and  non-pictorial 
materials.    In  one  study,  combinatorial  cues  (text  and  arrows)  benefitted  both  above 
and  below  average  reading  ability  groups.    This  suggests  that  in  addition  to 
identifying  labels,  arrows  used  to  portray  a  pattern  of  arrangement  or  order  of 
connection  as  parts  of  PSSs  in  the  human-computer  interface  could  benefit  viewers 
with  varied  levels  of  experience  related  to  computers. 

The  knowledge  gleaned  from  educational  media  research  often  applies  to  the 
design  of  the  human-computer  interface.    Designers  of  computer-assisted  instruction 
have  been  urged  to  follow  guidelines  developed  for  text  design  in  developing  screen 
displays  (Duin,  1988).    With  purely  textual  interfaces,  this  may  be  appropriate. 
Designers  need  to  be  careful  in  extending  pictorial-related  research  to  computers, 
since  recent  results  have  occasionally  appeared  to  contradict  previous  findings. 
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The  use  of  PSSs  in  the  human-computer  interface  often  differs  from  traditional 

uses  of  pictures  in  instruction.   Traditionally,  pictures  accompany  text;  the 
instructional  message  is  often  embodied  in  text  and  merely  supplemented  by  the  visual 
display.   PSSs  in  the  human-computer  interface  are  sometimes  displayed  without  text 
labels,  are  used  as  commands  to  enable  the  user  to  perform  a  task,  and  are  messages 
(instructional  or  informational)  in  themselves.    In  this  respect,  these  PSSs  are  related 
to  pictographic  and  public  information  symbols.    The  literature  in  this  area  should 
contribute  to  an  understanding  of  PSSs  used  in  human-computer  interfaces. 

Signs  and  Symbols.  Pictograms  and  Pictographs 

In  a  preliterate  or  illiterate  culture,  pictorial  signs  served  to  locate  services  in 
towns,  such  as  barbers'  or  bakers'  shops,  to  the  stranger  or  traveler.    In  the  modem 
world  of  global  transportation,  many  people  find  themselves  in  locations  where  they 
are  linguistically  illiterate,  and  must  rely  on  PSSs  to  locate  baggage,  money 
exchanges,  or  restrooms  (Modley,  1966).    International  trade,  travel,  and 
communication  have  created  a  need  for  standardization  of  symbols  and  proposals  for 
universal  graphic  symbol  systems  (Collins,  1982;  Dreyfuss,  1966;  Easterby,  1970; 
Modley,  1966). 

The  modem  attempt  to  develop  this  visual  mode  of  communication  began  with 
traffic  signs,  public  information  signs,  and  instrument  controls  before  the  development 
of  graphical  user  interfaces  for  computers.   Kolers  (1969)  recommended  that  PSSs  use 
realistic  pictures  of  technological  objects,  rather  than  analogical  pictures  of  naturally 
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occurring  objects,  on  the  assumption  that  different  cultures  may  interpret  natural 

objects  differently,  but  new  technology  is  used  consistently  across  cultures.    For 
example,  a  picture  of  a  telephone  would  be  universally  understood  to  anyone  familiar 
with  the  use  of  a  telephone,  while  a  picture  of  a  penguin  might  elicit  different 
interpretations  from  a  stock  person  in  a  grocery  store  and  an  ornithologist  in  a  zoo. 

Using  the  symbol  set  designed  by  Dreyfuss  (1966)  for  Deere  &  Co.,  Cahill 
(1975)  tested  whether  the  visual  context  of  the  symbol  and  prior  experience  of  the 
viewer  affected  the  interpretability  of  sign  symbols.    Thirty  mechanical  engineering 
majors  were  presented  with  a  set  of  ten  symbols  used  to  label  parts  of  heavy 
automotive  machinery  and  asked  to  name  the  part  or  function  each  represented.   Half 
of  the  subjects  were  provided  with  a  numbered  drawing  indicating  the  positions  of  the 
symbols  and  the  corresponding  instruments  within  a  cab  interior.    The  results 
supported  the  view  that  context  contributed  to  the  meanings  inferred  from  symbols. 
Prior  experience  with  the  same  type  of  machinery  also  made  some  difference. 

Cahill  made  no  distinctions  regarding  the  functions  of  the  pictorial  symbols. 
Analysis  of  the  stimuli  using  Knowlton's  taxonomy  yields  interesting  results,  but  no 
specific  pattern.   The  symbol  for  the  choke  function  was  a  realistic  sign  symbol  of  a 
butterfly  valve,  the  symbol  for  the  "engage"  function  was  a  logical  sign  symbol. 
Neither  of  these  two  symbols  were  interpreted  correctly.    A  realistic  sign  symbol  for 
the  hand  brake,  an  analogical  sign  symbol  for  the  car  horn,  and  a  logical  sign  symbol 
for  the  turn  signals  were  all  interpreted  correctly  regardless  of  prior  experience. 
These  findings  run  counter  to  general  guidelines  that  PSSs  for  concrete  objects  are 
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more  easily  interpreted  than  abstract  concepts,  as  well  as  Koler's  guideline  that 

technological  objects  should  be  interpreted  more  consistently  than  other  types  of 

representations. 

PSSs  used  in  automotive  controls  were  the  materials  for  a  study  conducted  by 
Green  and  Pew  (1978).   In  this  study,  subjects  were  drawn  from  a  pool  and  did  not 
represent  a  special  population.    Given  an  imaginary  scenario  to  provide  a  context  and 
provided  with  the  complete  set  of  19  symbols,  the  subjects  were  asked  to  point  to  a 
specific  symbol.    Only  six  of  the  symbols  met  an  acceptance  criterion  set  at  75% 
correct  responses.    Of  these,  four  were  realistic  sign  symbols  for  the  seat  belt,  front 
and  rear  hoods,  ventilating  fan,  and  the  battery  representing  the  charging  circuit.   The 
analogical  sign  symbol  of  a  bugle  referring  to  the  car  horn,  and  a  logical  sign  symbol 
consisting  of  the  mnemonic  letter  "P"  with  lines  representing  light  rays  emanating 
from  it  (signifying  the  parking  lights)  were  also  easily  comprehended.    A  sex  effect 
(males  interpreted  symbols  better  than  females)  and  a  sex  x  technical  training 
interaction  (male  students  majoring  in  engineering  interpreted  symbols  better  than 
males  majoring  in  non-technical  disciplines)  was  observed.    The  sex  effect  was 
spurious,  as  the  higher  overall  performance  of  males  was  attributed  to  those  with 
technical  training. 

Mackett-Stout  and  Dewar  (1981)  studied  different  public  information  signs 
intending  to  convey  the  same  message.    However,  there  was  no  systematic  variation 
of  symbols  to  provide  specific  categorical  exemplars  for  each  message.    They 
concluded  that  abstract  symbols  are  much  more  difficult  to  comprehend  than  iconic 
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pictographs.    In  contradiction  to  their  conclusion,  however,  the  logical  sign  symbol  of 

a  question  mark  representing  the  concept  of  "Information"  was  chosen  as  the  best 

overall  symbol  for  that  message. 

The  use  of  pictographic  materials  to  communicate  instructions  for  equipment 

operation  was  the  subject  of  experiments  conducted  with  an  artificial  piece  of 

equipment  (Marcel  &  Barnard,  1979)  to  explore  how  people  comprehend  and  use 

pictorial  instructions.   The  task  given  to  subjects  was  to  view  a  set  of  pictorial 

instructions  and  then  either  to  describe  the  instructions  verbally  so  someone  at  the 

other  end  of  a  telephone  could  follow  them  correctly  or  to  perform  the  instructions  on 

the  equipment.   Instructions  that  showed  the  entire  apparatus  were  easier  to 

comprehend  than  those  that  showed  only  the  relevant  parts  to  be  acted  on  at  a 

particular  step.   The  attempt  to  focus  attention  by  excluding  irrelevant  parts  was 

overridden  by  the  loss  of  the  overall  frame  of  reference  and  visual-spatial 

relationships  of  the  parts.   Fewer  relational  misinterpretations  occurred  when  the 

subjects  actually  operated  the  equipment.   Marcel  and  Barnard  noted  that  people 

tended  to  interact  with  the  equipment  in  a  problem-solving  manner,  drawing 

inferences  from  the  results  of  the  interactions,  instead  of  reading  the  instructions  first 

and  then  using  the  instructions  to  guide  their  actions.    They  also  indicated  that  people 

tended  to  operate  on  the  instructions  in  a  piecemeal  or  syntactic  fashion  and  only 

grasped  the  underlying  semantics  of  the  instructions  when  asked  to  verbalize  the 

instructions.    Marcel  and  Barnard  suggest  that  instructions  be  designed  based  on 

questions  that  users  have  for  particular  purposes  rather  than  on  a  task  analysis  of  how 
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the  equipment  works,  and  concluded  that  the  intelligibility  of  such  instructions  cannot 

be  assessed  outside  the  appropriate  context. 

In  summary,  studies  into  the  use  of  PSSs  in  the  form  of  pictographs  for 
language-free  communication  have  not  yielded  conclusive  recommendations  for 
general  application.   The  evidence  indicates  that  both  an  individual's  prior  knowledge 
and  experience  in  a  domain,  and  the  context  within  which  the  symbols  appear  affect 
comprehension.    The  focus  of  this  paper  now  shifts  to  the  human-computer  interface. 

Pictorial  Sign  Symbols  in  the  Human-Computer  Interface 

The  shift  in  concern  from  sheer  technological  power  to  human  factors  concerns 
of  ease  of  use  and  learning  occurred  in  computer  science  as  tremendous  technological 
advances  made  dramatic  cost  reductions  possible  (Shneiderman,  1984).   The 
expansion  of  computer  users  beyond  the  technically  trained  to  include  novices  and 
non-technically  trained  people  has  been  a  major  force  towards  the  concern  for  human 
factors  issues  in  the  human-computer  interface.    Increasing  dependence  on  interactive 
computer  systems  for  organizational  operations  and  communications  as  well  as  critical 
needs  for  error-free  operation  in  medicine,  air-traffic  control,  emergency  services, 
and  space  and  military  applications  have  also  been  important.    In  addition  to  computer 
science,  the  disciplines  of  cognitive  and  educational  psychology,  instructional  design, 
graphic  design,  technical  writing,  and  human  factors  are  all  involved  in  the  discovery 
of  solutions  to  problems  of  human  performance  in  interaction  with  computer  systems 
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(Shneiderman,  1987).    Systems  involving  direct  manipulation  of  PSSs  are  a 

manifestation  of  this  effort. 

The  use  of  PSSs  or  icons,  as  they  are  usually  referred  to  when  used  in  the 
human-computer  interface,  grew  out  of  human  factors  research  in  man-machine 
interfaces.   The  term  "icon"  as  originally  used  in  the  computer  interface  was  an 
analogy  to  religious  icons,  which  were  considered  to  embody  the  properties  of  the 
saints  they  represented  (Perry  &  Voelcker,  1989).    Icons  in  the  computer  interface 
were  treated  as  objects  that  contained  all  the  properties  and  attributes  of  the  programs 
and  data  represented,  and  could  therefore  be  acted  on  as  if  they  were  the  actual 
programs  or  data.   For  example,  the  act  of  dragging  a  file  into  a  trash  can  on  the 
computer  screen  represented  the  act  of  deleting  that  file.   A  more  recent  development 
has  been  the  use  of  PSSs  to  represent  commands  or  functions  or  menus.    Choosing 
(through  pointing  and  clicking)  a  command  would  carry  out  that  command  on  any 
selected  items  on  the  screen.    Although  the  functions  of  these  graphics  may  vary,  the 
term  "icon"  has  become  commonplace,  and  is  used  in  the  references  cited  below. 
Throughout  this  section,  the  term  "icon"  will  be  used  interchangeably  with  the  term 
"pictorial  sign  symbol"  to  avoid  redundancy. 

The  application  of  design  principles  to  a  computerized  office  system  led  to  the 
creation  of  the  Xerox  "Star"  system  (Smith  et  al.,  1982).    Human  factors  testing  of 
the  Star  interface  icons  included  paper  and  pencil  naming  tests,  timed  tests  to  measure 
recognizability  and  distinguishability,  and  ratings  tests  to  obtain  subjects'  opinions 
(Bewley,  Roberts,  Schroit,  &  Verplank,  1987).   The  icons  tested  were  all  objects, 
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such  as  document,  file,  and  printer;  icons  representing  commands  were  not 

incorporated  into  the  desktop  metaphor.    In  Knowlton's  taxonomy,  the  entire  desktop 

display  and  its  contents  would  be  considered  as  analogical  sign  symbols.    Subsequent 

graphical  user  interfaces  have  been  criticized  for  imitation  of  the  desktop  metaphor 

without  adequate  definition  of  intended  users  and  appropriate  interface  features 

(Gittens,  1986;  Nadin,  1988). 

Lodding  (1983)  proposed  three  characteristics  of  an  optimal  icon.    First,  the 
viewer  should  be  able  to  infer  its  meaning  on  the  first  encounter.    Second,  when 
selecting  from  a  menu  or  set  of  choices,  only  one  icon  should  be  appropriate;  and 
third,  the  icon  should  not  convey  any  negative  connotations. 

An  example  of  an  interface  that  goes  beyond  the  desktop  metaphor  is  that  of 
the  "book  house"  in  which  users  may  search  for  books  by  interacting  with  icons  as  an 
alternative  to  the  traditional  keyboard  based  command  dialog  system  interface  seen  in 
typical  library  automation  systems  (Pejtersen  &  Goodstein,  1990).   The  metaphor 
chosen  for  this  database  was  that  of  a  "book  house."   After  a  user  chooses  to  interact 
with  a  mouse  input  device,  a  screen  appears  that  shows  three  rooms  with 
representations  of  two  children,  an  adult  and  a  child,  and  two  adults,  respectively, 
standing  in  front  of  the  rooms.    At  this  point  the  user  can  choose  to  work  with  the 
children's  book  database,  the  total  database,  or  the  adult  books  database  by  clicking 
the  mouse  in  the  appropriate  area.    Upon  entering  a  room,  representations  of  people 
performing  various  search  strategies  are  shown  on  the  display  to  indicate  the  search 
strategy  alternatives  available.    The  types  of  searches  available  are  analytical  search, 
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search  by  analogy,  browsing  in  pictures,  and  browsing  in  book  descriptions.  The 

analytical  search  provides  indices  to  the  user,  in  the  form  of  an  open  book.   The 

search  by  analogy  allows  the  user  to  select  a  book  title  and  then  searches  for  books 

that  are  similar  to  the  one  chosen.   The  picture  browse  search  allows  the  user  to  pick 

PSSs  that  appear  interesting,  and  then  presents  titles  within  that  topic.   The  book 

description  browse  provides  random  selected  book  descriptions  from  the  data.    With 

each  of  these  types  of  searches,  Boolean  logic  can  be  applied. 

Testing  for  meaningfulness  of  icons  used  in  this  interface  involved  trained 

librarians,  adult  users,  and  children.    The  pictorial  symbols  were  each  displayed  for 

25  seconds,  followed  by  a  list  of  several  intended  subject  areas.    The  subjects  then 

identified  the  concept  communicated  by  the  icon  from  the  list  of  intended  subject 

areas.    Although  specific  data  are  not  reported,  the  authors  indicate  that  some  icons 

were  interpreted  strongly  as  only  one  of  the  terms  provided,  others  were  interpreted 

in  a  more  varied  manner.    Icons  that  directly  represented  the  object  signified  were 

easily  interpreted.    Children's,  and  to  a  lesser  degree,  women's  responses  were  more 

widely  distributed  than  men's  responses.    Due  to  these  results,  the  search  terms  used 

in  the  children's  book  database  for  the  same  PSS  are  different  from  those  used  in  the 

adult  book  database  (Pejtersen  &  Goodstein,  1990).   Caution  must  be  used  in  drawing 

any  conclusions  from  this  report,  as  it  provides  examples  only,  and  lacks  any 

evaluative  data  from  actual  use.    Another  limitation  in  drawing  conclusions  from  this 

and  other  studies  is  that  direct  representation  of  a  concrete  object,  as  often  described, 

does  not  tell  us  whether  the  visual  representation  functions  as  a  realistic  or  an 
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analogical  sign  symbol.   It  is  interesting  to  see  the  possibilities,  potential,  and 

problems  in  applying  a  complete  metaphor  involving  PSSs  (other  than  the  desktop 

metaphor)  to  the  human-computer  interface  for  an  information  processing  application. 

Apple  Computer  has  made  a  strong  effort  to  incorporate  PSSs  into  its 

Macintosh  desktop  interface  metaphor  in  a  consistent  manner  (Apple  Human  Interface 

Guidelines,  1987;  Nadin,  1988).   Recent  studies  conducted  by  the  Apple  Human 

Interface  Group  (Baecker,  Small,  &  Mander,  1991)  indicate  that  some  currently  used 

icons  are  less  than  optimal.    Although  the  main  focus  of  their  study  was  to  explore 

the  use  of  animated  icons,  several  static  icons  already  in  use  were  consistently 

misinterpreted  by  the  subjects,  notably  the  "paint  bucket"  tool.   This  tool's  function  is 

to  fill  an  enclosed  area  with  a  selected  pattern.   Clear  differences  were  seen  between 

experienced  and  novice  users;  however,  the  limited  number  of  subjects  and 

exploratory  nature  of  the  study  limit  conclusions  to  be  drawn  from  this  study.    The 

paint  bucket  tool,  interpreted  incorrectly  by  all  novice  subjects,  is  a  "concrete"  image 

in  most  classifications.    Generally,  concrete  images  and  messages  are  recommended 

for  novices  (Fleming  &  Levie,  1978).    Although  it  is  an  iconic  representation  of  a 

paint  bucket,  its  function  is  analogical,  in  that  the  computer  function  that  it  refers  to 

produces  an  effect  that  is  analogous  to  the  effect  produced  by  pouring  paint  onto  a 

surface.    In  Knowlton's  taxonomy  the  paint  bucket  tool  would  be  classified  as  an 

analogical  picture.    An  adequate  theoretical  model  to  describe,  explain,  and  predict 

the  meaningfulness  of  PSSs  would  help  to  deal  with  data  that  apparently  contradicts 

earlier  findings. 
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Researchers  into  the  human-computer  interface  have  often  used  technically 

trained  subjects  (Guastello,  Traut,  &  Korienek,  1989;  Israelski  et  al.,  1989;  Tullis, 
1981),  which  can  lead  to  concerns  about  the  validity  and  generalizability  of 
experimental  results  to  other  populations,  especially  novices.    Even  with  technical 
experts,  different  studies  have  produced  disparate  results.    Some  populations  may 
prefer  text  codes  to  pictorial  symbols  or  may  prefer  newly  created  symbols  to  industry 
standard  conventions  (Guastello,  Traut,  &  Korienek,  1989).    On  the  other  hand, 
studies  have  also  found  technical  experts  both  preferred  and  improved  performance 
with  a  graphic  interface  through  direct  manipulation  of  icons  (Israelski  et  al.,  1989; 
Tullis,  1981).    Contrary  results  have  been  reported;  under  some  task  conditions,  icons 
in  the  human-computer  interface  provided  no  performance  advantages  (Landsdale, 
1988). 

Some  investigations  into  aspects  of  the  human-computer  interface  have 
provided  support  for  the  use  of  principles  and  guidelines  from  the  field  of 
instructional  message  design.   These  will  be  briefly  noted  below. 

Material  that  is  meaningful  to  the  learner  is  acquired  and  retained  more  easily 
(Fleming  &  Levie,  1978).   Icons  that  are  meaningful  to  users  are  recalled  more 
effectively  and  improve  performance  in  the  human-computer  interface  (Landsdale, 
1988). 

Landsdale  coded  icons  by  shape,  location  on  screen,  and  color  and  provided 
subjects  with  a  simulated  filing  task  for  information  storage  and  retrieval.  Subjects 
had  to  assign  job  advertisements  to  an  employment  database  using  icons,  then 
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retrieved  the  advertisements  following  a  one  hour  delay.    Iconic  methods  alone  did 

not  automatically  improve  performance.    Meaningfulness  of  icons  and  representational 

shapes  improved  both  recall  and  performance.    For  instance,  an  advertisement  related 

to  educational  television  would  be  assigned  for  filing  under  an  icon  of  a  person 

wearing  a  mortarboard  cap  and  gown.   In  a  meaningless  condition,  the  icon  would  be 

arbitrarily  assigned  by  the  system  to  a  shape  that  was  not  compatible  with  the 

advertisement.    Landsdale  also  found  that  subjects'  generation  of  the  encoding  icons 

used  for  filing  improved  performance.    For  example,  under  this  condition  the  subject 

might  choose  to  file  an  advertisement  relating  to  educational  television  under  an  icon 

representing  tragedy-comedy  masks  (for  entertainment)  instead  of  the  mortarboard 

icon. 

Pictures  are  more  easily  remembered  than  words  (Fleming  &  Levie,  1978). 
Menu  selections  with  graphics  are  more  accurately  searched  for  information  than  text 
menus  (Muter  &  Mayson,  1986). 

Muter  and  Mayson  contrasted  menu  selection  using  a  linear  text  menu,  a 
graphic  menu  with  the  same  text  items  with  related  graphics  spread  across  the  screen, 
and  a  control  menu  with  the  text  items  in  the  same  location  as  in  the  graphics  screen, 
but  minus  the  graphics.    For  instance,  a  menu  item  titled  "2.  Furniture"  was 
presented  in  the  center  of  the  screen  with  an  outline  drawing  of  a  table  and  chairs.    In 
the  control  condition  the  graphic  would  be  absent.    In  the  text  condition  it  would 
appear  second  in  order  in  the  list  of  menu  options.    Slides  of  the  screens  were 
displayed  and  subjects  were  asked  to  respond  to  goal  oriented  questions  or  commands 
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related  to  the  menu  items.    For  example,  "Look  for  a  new  sofa"  should  have  resulted 

in  a  response  of  "2.  Furniture."   Though  there  was  no  difference  in  speed  overall,  the 

graphics  menus  resulted  in  fewer  errors. 

Combined  text  and  graphics  should  be  congruent  or  redundant,  so  the  learner 
receives  the  same  message  regardless  of  the  channel  of  communication  attended  to 
(Fleming  &  Levie,  1978).    Mixed  modality  icons  (text  and  graphics)  are  more 
meaningful  and  preferred  by  users  within  a  content  domain  (Guastello,  Traut,  & 
Korienek,  1989). 

A  survey  study  providing  several  alternatives  to  represent  system  controls  and 
displays  in  an  automated  building  heating,  ventilation,  and  air-conditioning  system 
was  conducted  by  Guastello  and  associates  (1989).   Their  data  supported  the 
hypothesis  that  dual-modality  icons  would  be  more  meaningful  than  either  modality  by 
itself.    Longer  verbal  abbreviations  were  preferred  over  shorter  industry  standard 
abbreviations.   Perhaps  due  to  the  technical  background  of  the  population  surveyed, 
verbal  icons  were  sometimes  preferred  over  pictorial  if  the  mixed  modality  icon  was 
not  available. 

Discriminations  are  learned  most  readily  by  progressing  from  maximal 
differences  to  finer  differences  (Fleming  &  Levie,  1978).   Menu  navigation  is 
facilitated  by  maximal  distinctiveness  among  text  choices  (Schwartz  &  Norman, 
1986).    Visual  distinctiveness  of  PSSs  aids  in  locating  specific  items  (Arend,  Muthig, 
&  Wandmacher,  1987). 
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Schwartz  and  Norman  (1986)  modified  an  existing  menu  used  by  the 

CompuServe  time-sharing  system.    Menu  modifications  consisted  primarily  of  adding 

the  words  "General"  or  "Specific"  to  the  beginning  of  a  menu  item  and  the  word 

"Information"  at  the  end  of  the  item.   This  "rather  trivial"  manipulation  resulted  in 

significantly  modified  performance.   Subjects  using  the  modified  menus  to  locate 

information  within  the  system  performed  better  than  those  using  the  original  menu 

along  the  dimensions  of  time  per  search,  number  of  frames  covered  during  search, 

and  number  of  searches  quit.   Subjects  with  written  documentation  accompanying  the 

original  menu  quit  searching  more  often;  the  presence  of  descriptions  of  the  menu 

items  served  to  discourage  subjects  rather  than  facilitate  searches. 

Noting  that  research  such  as  that  described  above  dealt  only  with  text  items 

and  not  with  PSSs,  Arend,  Muthig,  and  Wandmucher  (1987)  investigated  commands 

using  a  word  set,  an  abstract  icon  set,  and  a  representational  icon  set.    Icons  in  the 

abstract  set  would  be  categorized  as  logical  pictures  in  Knowlton's  taxonomy.   These 

items  differed  greatly  in  overall  shape  or  global  features.    The  representational  icon 

set  would  be  categorized  as  analogical  pictures,  in  that  the  primary  element  was  a 

representation  of  a  sheet  of  paper.   In  this  set,  local  features  differed  while  the  global 

feature  of  the  sheet  of  paper  remained  constant.   Menus  featuring  icons  with  global 

differences  were  searched  more  quickly  than  text  or  local  difference  icons.    Increasing 

menu  size  from  6  to  12  items  did  not  significantly  increase  search  time  for  icons  with 

global  differences,  but  did  for  text  and  icons  with  local  differences.    The  abstract 
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icons  used  in  this  experiment  were  retrieved  more  quickly  regardless  of  menu  size 

without  any  difference  in  error  rates. 

Although  some  principles  for  instructional  message  design  have  been  supported 
by  research  involving  human-computer  interfaces,  studies  have  also  yielded 
contradictory  results.    This  evidence  will  be  noted  next. 

Concreteness  facilitates  learning  and  remembering.    Messages  to  the  novice 
within  a  domain  should  begin  with  the  more  concrete  and  move  to  the  more  abstract 
as  the  learner  increases  knowledge  (Fleming  &  Levie,  1978).     The  assumption  that 
icons  should  represent  concrete  objects  is  not  supported  in  some  human-computer 
interface  experiments  (Arend,  Muthig,  &  Wandmacher,  1987;  Baecker,  Small,  & 
Mander,  1991;  Blankenberger  &  Hahn,  1991). 

As  noted  above,  Arend,  Muthig,  and  Wandmacher  found  that  the  more 
concrete  and  representational  icons  were  not  searched  more  quickly  or  accurately  than 
were  those  that  were  more  abstract  in  representation.    Blankenberger  and  Hahn  found 
that  when  screen  placement  was  fixed,  the  design  of  the  menu  items,  whether  pictorial 
or  text,  made  no  significant  difference  after  learning.    However,  Blankenberger  and 
Hahn  also  found  that  the  closer  "articulatory  distance"  (Hutchins,  Hollan,  &  Norman, 
1986)  of  representational  icons  assisted  in  learning  to  associate  icons  with  their 
respective  commands  more  effectively  than  meaningless  arbitrary  figures. 

These  studies  indicate  that  users  can  adapt  to  varied  types  of  PSSs  within  the 
human-computer  interface  and,  given  training,  there  is  little  difference  in  search  and 
selection  performance  after  items  and  locations  have  been  memorized.   The  studies 
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cited  have  typically  involved  investigations  of  PSSs  contrasted  or  in  combination  with 

text,  and  have  explored  differences  between  visual  and  verbal  memory  processes. 

Generally,  the  primary  focus  has  not  been  to  determine  the  functional  intent  or  initial 

interpretation  of  the  meaning  of  the  PSS.   As  noted  earlier,  only  recently  have  icons 

been  used  as  commands  in  addition  to  representing  data  or  programs.    Therefore,  the 

discussion  will  focus  next  on  specific  studies  that  have  explored  the  construction  of 

PSSs  for  use  as  commands  and  will  explore  the  rationale  for  addressing  this  as  an 

instructional  message  design  problem  using  Knowlton's  taxonomy. 

Gabriele  Rohr  (Rohr,  1984,  1986;  Rohr  &  Keppel,  1984)  conducted  a  series  of 
studies  to  investigate  the  relationship  of  types  of  commands  to  types  of  pictorial 
symbols  to  determine  whether  commands  could  be  successfully  represented  and 
understood  using  pictorial  symbols  rather  than  through  verbal  commands  alone. 
Types  of  commands  included  those  that  dealt  with  manipulating  text,  processing  files, 
and  controlling  internal  states.   Text  manipulation  commands  included  items  such  as 
input,  insert,  move,  and  delete.    File  processing  commands  included  items  such  as 
save,  send,  erase,  and  print.    State  control  commands  included  select,  help,  and  back. 

Icon  command  sets  were  developed  by  showing  pictorial  and  abstract  elements 
singly  and  in  combinations  to  subjects.    Pictorial  symbols  used  included  iconic 
representations  of  sheets  of  paper,  pencil,  a  hand,  a  tear  in  the  paper,  an  envelope,  a 
hand  stamp,  and  a  box.    Abstract  symbols  included  such  elements  as  rectangles, 
slashes,  crosses,  arrows,  points,  letters,  and  numbers.    A  group  of  ten  subjects  with 
no  prior  data  or  word  processing  experience  rated  each  item  in  relation  to  the  amount 
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of  functionality,  manipulative  information,  or  pictorial  quality  contained  in  each  item. 

They  also  rated  how  well  icons  represented  text  processing  functions. 

The  arrow  was  the  only  icon  element  rated  highly  for  functionality  and 

manipulative  information.   File  processing  commands  were  found  to  be  best 

represented  using  pictorial  symbols  alone  or  in  combination  with  abstract  symbols 

such  as  arrows.    For  text  manipulation  and  state  control,  abstract  symbols  indicating 

actions  were  chosen.    Based  on  the  ratings  provided  by  these  subjects,  three  sets  of 

icons  were  constructed  using  the  elements  identified. 

These  icon  command  sets  included  two  pictorial  sets  and  one  word  set.   The 

two  pictorial  sets  were  varied  systematically  to  include  or  exclude  functional 

elements.    For  example,  the  papersheet  pictorial  symbol  was  selected  as  symbolizing 

"file."   This  was  combined  with  other  symbols,  such  as  the  "tear"  symbolizing 

"erase"  and  "delete"  to  create  all  file  processing  commands.    Two  sets  of  file 

processing  commands  were  created  that  included  either  an  arrow  (functional  symbol) 

symbolizing  an  action,  or  lacking  an  arrow  (non-functional  symbol).   Text  editing  and 

state  control  icons  were  created  using  similar  techniques,  except  that  text  editing 

commands  showed  shaded  rectangles  denoting  cursor  placement  actions  for  the 

functional  symbol  set.    The  state  control  icons  used  arrows  to  indicate  the  selected 

state.    These  sets  were  incorporated  into  a  prototype  text  editing  system  (Keppel  & 

Rohr,  1984)  that  differed  only  in  the  icon  sets  displayed  for  the  commands. 

Following  minimal  instruction,  subjects  with  no  prior  text  processing  experience  were 

given  tasks  to  perform  using  the  system. 
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Results  of  these  studies  indicated  a  superiority  of  visual  symbols  over  verbal 

commands  in  the  initial  learning  stage,  as  indicated  by  significantly  more  help 

requests  from  those  learning  the  verbal  command  group.   The  second  part  of  the 

experiment  showed  no  difference  between  groups  in  the  number  of  help  requests.    In 

addition  to  number  of  help  requests,  Rohr  also  measured  time  in  editing.  Functional 

symbol  groups  took  longer  to  learn,  but  were  superior  to  both  non-functional  symbol 

and  verbal  command  groups  during  the  second  part  of  the  experiments.    Rohr 

indicated  that  the  visual-spatial  information  encoded  in  the  functional  text 

manipulation  symbols  helped  to  reduce  mental  memory  workload  by  presenting  visual 

cues  regarding  the  operation  of  the  commands. 

Rohr  also  conducted  experiments  using  paper  and  pencil  testing  as  a  control, 

after  providing  instruction  using  the  help  screens  from  the  iconic  command  version 

and  the  word  command  version  (Rohr,  1986).   This  time,  the  functional 

representation  icons  differed  only  with  spatial-relations  tasks.    There  was  a  learning 

effect,  as  the  functional  icons  took  longer  to  learn,  but  were  associated  with  faster 

performance  during  the  second  measurement  of  task  performance.   Rohr  concluded 

that  the  nature  of  the  task  determined  the  utility  of  the  icon  commands.   Subjects 

performed  tasks  that  involved  spatial  relationships  more  effectively  using  the 

functional  icon  commands  after  learning.    File  handling  iconic  commands  had  no 

advantage  over  word  commands.    Visual  information  was  advantageous  when  the  task 

involved  spatial  functions,  by  reducing  memory  workload  and  reducing  the  complexity 
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of  the  task.    With  more  highly  abstract  functions,  visual  concepts  increased 

complexity  and  were  not  recommended. 

An  analysis  of  the  iconic  commands  used  by  Rohr  in  this  study,  using 
Knowlton'  s  taxonomy,  shows  that  the  file  handling  commands  were  analogical 
pictures  showing  objects  and  places,  and  that  the  text  editing  commands  were  logical 
pictures.    Since  Rohr  developed  the  icon  sets  before  implementation,  it  cannot  be  said 
that  analogical  pictures  are  better  for  a  certain  type  of  command  than  either  realistic 
pictures  or  logical  pictures.    Analysis  of  individual  PSSs  reveals  that  the  pattern  of 
arrangement  or  order  of  connection,  which  Rohr  considered  spatial  functions,  were 
consistently  represented  using  arrows.    Due  to  the  findings  by  Rohr,  the  PSS  sets 
constructed  for  this  study  include  arrows  as  components  of  realistic,  analogical,  and 
logical  pictures  alike.   The  addition  of  arrows  or  functional  indicators  would  seem  to 
provide  additional  information  that  would  be  difficult  to  infer  without  the  arrows. 
And,  in  the  small  sample  used  to  identify  meaningful  elements,  the  arrow  was  the  one 
item  that  was  meaningful  to  all  viewers  in  relation  to  function. 

Building  on  Rohr's  work,  and  using  a  classification  system  with  some  elements 
similar  to  Knowlton's  taxonomy,  Rogers  (1986)  investigated  the  relationship  between 
type  of  representation  and  the  initial  comprehension  of  icon  sets  used  to  represent 
word  processing  command  operations.    Rogers'  symbols  were  classified  by  type  as 
"abstract  symbol,"  "concrete  analogy  associated  with  action,"  and  "concrete  object 
operated  on."   These  simple  sets  were  used  to  create  an  additional  three  combined 
symbol  sets. 
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Rogers  provided  subjects  with  a  verbal  context  appropriate  to  word  processing 

tasks.   The  full  icon  set  was  provided  with  descriptions  of  the  commands  included. 

The  subjects'  task  was  to  match  the  icons  to  the  command  descriptions.    Subjects 

were  advised  to  cross  out  each  icon  after  having  matched  it  to  a  description.    Main 

effects  by  command  set  were  found,  as  well  as  interactions  between  symbol  type  and 

command  type.   The  conclusion  was  made  that  icons  representing  concrete  objects  to 

be  acted  on  were  superior  for  use  as  commands. 

However,  Rogers'  icons  include  abstract  symbols  as  elements  and  as  indicators 

of  pattern  or  order  of  connection  without  making  a  distinction  between  the  different 

functions.    Rogers'  definition  of  concrete  objects  and  concrete  analogies  did  not 

consider  the  analogical  nature  of  the  desktop  metaphor  and  its  elements.    For 

example,  a  representation  of  a  sheet  of  paper  with  a  small  arrow  at  the  bottom  was 

considered  a  concrete  object.    A  finger  pointing  downward  was  considered  an 

analogy.    An  arrow  pointing  downward  was  considered  an  abstract  symbol.   The 

encoded  message  within  each  of  these  symbols  was  the  command  "Go  to  the  bottom 

of  the  text. "  The  combinatorial  symbols  all  contained  a  representation  of  a  sheet  of 

paper  as  a  concrete  object.   The  arrow  pointing  downward  was  added  as  an  abstract 

symbol,  and  the  hand  pointing  downward  was  added  as  a  concrete  analogy.   The 

combination  of  all  three  classifications  in  one  picture  was  achieved  by  using  the  sheet 

of  paper,  the  hand  pointing  downward  as  the  concrete  analogy,  and  the  addition  of 

motion  cues  in  the  form  of  two  lines  next  to  the  hand. 
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Rogers  concluded  that  the  use  of  depictions  of  concrete  objects  operated  on 

was  the  most  direct  and  meaningful  way  to  represent  abstract  referents.   Icons  that 

depicted  command  operations  by  her  definition  of  a  concrete  analogy  were  the  least 

effective.    Although  she  referred  in  her  introduction  to  Rohr's  (1984)  finding  that 

manipulative  control  actions  such  as  "insert  text"  were  best  represented  by  more 

abstract  symbols,  she  did  not  discuss  or  explain  this  apparent  discrepancy  with  Rohr's 

study.   One  possible  explanation  is  that  the  symbol  sets  used  by  Rohr  and  Rogers 

were  not  equivalent. 

Knowlton's  taxonomy  provides  a  framework  for  examining  Rogers'  sets  of 
PSSs  that  may  explain  the  results.    The  category  "concrete  objects  operated  on," 
which  included  the  best  understood  symbols,  includes  analogical  pictures  when 
categorized  by  Knowlton's  schema.    Rogers'  "concrete  analogy"  symbols  are 
analogical  pictures  in  Knowlton's  taxonomy,  and  her  "abstract  symbol"  category 
appears  similar  to  Knowlton's  logical  pictures  classification.    Of  the  symbols  included 
in  her  article  (Rogers,  1986),  a  depiction  of  a  blank  computer  screen  representing  the 
"quit"  command  is  the  only  command  that  might  be  considered  as  a  realistic  sign 
symbol  within  Knowlton's  taxonomy.    How  would  one  classify  Rogers'  other 
examples  more  parsimoniously? 

For  example,  consider  the  PSSs  used  by  Rogers  for  the  command  "go  to 
bottom  of  text."   The  text  displayed  on  the  screen  and  kept  in  electronic  form  in  the 
computer's  memory  is  represented  by  analogy  as  a  sheet  of  paper  with  text  on  it. 
Although  the  picture  might  be  considered  to  be  more  concrete  than  abstract  in  its 
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depiction  of  a  sheet  of  paper,  it  still  functions  as  an  analogy.    Since  this  sheet  of 

paper  is  the  main  element  of  the  picture,  the  PSS  should  be  classified  as  an  analogical 

picture.   The  addition  of  arrows  or  pointing  fingers  denoting  the  goal  of  the  command 

"go  to  bottom  of  text"  indicate  only  the  arrangement  or  connection  of  the  current 

cursor  location  and  the  destination  cursor  location  and  do  not  change  the  functional 

classification  within  the  taxonomy.   The  "concrete  object"  picture  used  to  denote  this 

command  actually  includes  an  arrow  pointing  at  the  bottom  of  the  text  and  could  be 

interpreted  as  equivalent  to  Rogers'  "concrete  object  plus  abstract  symbols" 

classification.    In  addition,  including  abstract  (logical)  components  within  the  symbols 

proved  superior  to  the  simple  icon  set.    This  might  be  explained  both  by  Rohr's 

results,  in  which  text  manipulations  were  more  effectively  represented  by  more 

abstract  visuals,  and  by  instructional  message  design  guidelines,  which  suggest  that 

concreteness  facilitates  learning  and  remembering,  concrete  pictures  are  more  easily 

remembered  than  words,  and  that  cues  that  are  familiar  to  the  learner  or  that  direct 

attention  to  relationships  can  facilitate  learning.    Although  Rohr  and  Rogers  provided 

additional  information  about  the  interpretation  of  PSSs  used  in  human-computer 

interfaces,  the  constructs  used  by  Rogers  were  poorly  defined.    Rohr's  primary 

purpose  was  to  develop  graphic  menu  commands  rather  than  to  test  or  develop  theory. 

This  study  was  designed  to  explore  questions  left  unanswered  by  both  Rohr's 

and  Rogers'  studies  regarding  the  initial  comprehension  and  interpretation  of  PSSs 

used  in  the  human-computer  interface.    In  this  investigation,  Knowlton's  taxonomy 

was  used  systematically  to  create  sets  of  PSSs,  so  each  command  was  represented  by 
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an  example  from  each  functional  type  of  picture.    This  counterbalancing  was  not  done 

in  Rohr's  studies.    Knowlton's  taxonomy  offers  a  more  concise  scheme  for 

classification  than  does  Rogers'  categories.    As  discussed  above,  Rogers'  schema  is 

not  as  comprehensive  as  Knowlton's  in  describing  and  explaining  all  possible 

combinations.   The  inclusion  of  arrows  as  a  component  within  PSSs  aids  in 

comprehension;  in  this  study,  the  pattern  of  arrangement  and  order  of  connection 

within  a  PSS,  where  appropriate,  were  represented  consistently  by  arrows  across 

functional  types. 

Computer  Experience.  Literacy,  and  Knowledge 

Researchers  have  used  experts  and  novices  (Baecker,  Small,  &  Mander,  1991; 
Cahill,  1975)  as  categories  in  investigating  interpretation  of  PSSs.    Cahill  found  that 
subjects  with  specific  experience  with  the  type  of  equipment  represented  were  able  to 
interpret  PSSs  that  referred  to  that  equipment's  controls  more  successfully  than 
subjects  with  equal  technical  knowledge  but  lacking  the  specific  experiential  context. 
The  study  conducted  by  Baecker  and  associates  distinguished  between  subjects  with  no 
experience  and  subjects  with  prior  experience  with  the  software  program  used  in  the 
study.    Computer  experience  and  knowledge  will  vary  among  individuals,  and  can  be 
expected  to  have  an  effect  on  the  interpretation  of  PSSs  used  in  the  human-computer 
interface. 

For  the  purposes  of  this  study,  two  nationally  standardized  instruments  found 
in  the  Educational  Testing  Service  Catalog  were  reviewed  for  suitability.   These 
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measures  are  the  Computer  Aptitude,  Literacy,  and  Interest  Profile  (CALIP)  (Poplin, 

Drew,  &  Gable,  1984),  and  the  Standardized  Test  of  Computer  Literacy  (Montag, 

Simonson,  &  Maurer,  1984). 

The  CALIP  is  designed  to  assess  computer-related  abilities  of  individuals 
between  12-60  years  of  age.    The  CALIP  is  a  standardized  test  battery  composed  of 
four  aptitude  subtests,  one  interest  subtest,  and  one  literacy  subtest.    The  four  aptitude 
subtests  (Estimation,  Graphic  Patterns,  Logical  Structure,  and  Series)  are  primarily 
visual  in  nature  and  do  not  require  much  reading.    Standard  scores  from  the  four 
aptitude  subtests  are  combined  and  a  Computer  Aptitude  Quotient  (CAQ)  is  derived 
from  a  conversion  table.   The  CALIP  can  be  administered  individually  or  in  groups, 
and  requires  approximately  one  hour  for  completion. 

The  CALIP  was  standardized  on  a  national  sample  of  1,236  children, 
adolescents,  and  adults.    Levinson  (1986)  notes  that  three  of  the  four  aptitude  subtests 
are  highly  correlated  with  each  other,  and  may  measure  the  same  construct,  which 
may  be  nonverbal  problem-solving  ability.    Validity  studies  reported  in  the  manual 
indicate  that  the  CALIP  can  differentiate  between  expert  programmers  and  random 
others,  between  those  who  have  written  intermediate/advanced  programs  and  random 
others,  and  between  those  who  know  two  or  more  computer  languages  and  those  who 
know  one  or  none.    Significant  differences  are  also  reported  between  those  who  have 
had  two  or  more  computer  courses  versus  those  who  have  had  one  or  no  courses 
(Poplin  et  al.,  1984).    Levinson  (1986)  suggests  that  the  CALIP  can  differentiate 
between  individuals  with  greater  and  lesser  computer-related  experience,  but  not 
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necessarily  between  individuals  with  greater  or  lesser  aptitude  when  experience  is 

equal.    Internal  consistency  reliability  coefficients  are  above  .85  for  a  population 

roughly  equivalent  to  that  sampled  for  this  project.    Specifically,  the  Spearman-Brown 

split-half  reliability  coefficients  for  the  subtests  ranged  from  .86  to  .95  for  the  age 

groups  between  15  and  30  years  old.   The  coefficient  alphas  ranged  from  .85  to  .93. 

The  Standardized  Test  of  Computer  Literacy  (STCL)  was  developed  over  a 
two-year  period  at  Iowa  State  University  in  Ames,  Iowa  (Montag  et  al.,  1984).  The 
STCL  includes  a  competency  based  test  of  general  computer  literacy,  comprised  of 
three  parts,  and  a  Computer  Anxiety  Index  (CAIN).   The  three  sections  of  the  STCL 
include  competencies  related  to  computer  systems,  computer  applications,  and 
computer  programming.    Reliability  estimates  for  the  STCL  are  .64,  .75,  and  .69  for 
the  three  sections,  with  an  estimated  reliability  of  .86  for  the  total  STCL.    These 
estimates  were  based  on  a  pilot  test  of  152  college  students  who  had  just  completed  a 
semester  long,  three  credit,  college  level  computer  literacy  course.    Normative  data 
were  obtained  from  341  students  in  six  states,  with  an  estimated  reliability  of  .87 
(Simonson,  Maurer,  Montag-Torardi  &  Whitaker,  1987).   The  CAIN  was  found  to 
have  an  internal  consistency  reliability  estimate  of  .94  and  a  test-retest  reliability 
estimate  of  .90. 

The  CALIP  and  the  STCL  do  not  appear  to  be  equivalent  to  each  other.    The 
only  area  of  overlap  between  the  two  is  in  the  assessment  of  computer  literacy.    The 
CALIP  includes  a  30-item  subtest  of  computer  literacy  that  includes  questions 
intended  to  discriminate  between  those  with  high  and  low  levels  of  knowledge  and 
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experience.    One  question,  related  to  types  of  floppy  disks,  refers  to  disks  that  are 

single  sided  or  eight  inches  wide.   This  reveals  the  period  during  which  the 

instrument  was  developed,  and  suggests  that  this  particular  subtest  may  not  be 

currently  valid.   The  emphasis  on  graphic  materials  in  the  estimation  and  graphic 

patterns  subtests,  and  on  symbolic  data  in  the  logical  structures  and  series  subtests 

suggest  that  these  subtests  would  not  reflect  specific  computer  course  curricula.    A 

computer  experience  survey  is  included  with  the  CALIP,  but  is  not  part  of  the  scored 

items.   The  STCL,  on  the  other  hand,  uses  80  questions  to  measure  70  specific 

competencies  related  to  general  computer  literacy  in  the  areas  of  systems, 

applications,  and  programming.    A  single  question  about  a  technological  anachronism 

is  less  likely  to  affect  the  overall  results.    The  STCL  was  normed  on  students  who 

had  completed  a  semester  course  in  computer  literacy,  and  takes  1.5  hours  to 

administer.   The  scope  of  the  STCL  is  much  wider  than  was  needed  for  this  study. 

Since  the  purpose  of  this  study  was  to  investigate  the  effects  of  functional  type 
on  individuals'  ability  to  interpret  the  meanings  of  PSSs,  the  CALIP  was  considered 
to  be  more  closely  correlated  to  the  nonverbal,  visual  nature  of  the  stimuli  than  the 
competencies  measured  by  the  STCL.   Use  of  the  CALIP  could  confound  the  effect  of 
the  sign  symbols  with  non-verbal  problem  solving  ability.   This  would  make  the 
CALIP  an  unsuitable  choice  for  this  study. 

Because  no  suitable  nationally  standardized  instrument  was  practical  for  this 
study,  an  instrument  was  developed  locally  to  classify  subjects  by  type  of  computer 
related  experience.   This  instrument  consists  of  two  parts.   The  first  part  assesses  the 
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breadth  and  depth  of  computer  experiences.    The  second  part  of  the  instrument 

consists  of  a  self-assessment  of  expertise  and  experience.   Together,  these  two 

sections  are  used  to  arrive  at  a  composite  category.    This  instrument  was  tested  in  a 

pilot  project  to  develop  reliability  estimates  prior  to  the  implementation  of  the 

experimental  study.    A  discussion  of  the  computer  experience  survey  instrument  is 

included  in  the  next  chapter. 

Summary 

Pictorial  sign  symbols  used  in  a  human-computer  interface  may  be  viewed  as 
non-verbal  instructional  messages.    Selected  materials  from  the  theoretical  and 
research  literature  of  several  disciplines  have  been  reviewed  to  place  the  issues  related 
to  the  initial  interpretation  of  these  PSSs  in  perspective.    Theoretical  assumptions 
regarding  the  design  of  these  messages  have  not  been  supported  in  some  situations, 
generally  in  regard  to  the  level  of  concreteness  or  abstraction  of  the  PSS.    A 
semiotics-based  approach  has  been  suggested  and  has  been  used  for  dealing  with 
PSSs,  but  an  adequate  taxonomy  for  the  criterial  attributes  of  these  symbols  within  the 
human-computer  interface  has  not  yet  emerged. 

Knowlton's  taxonomy  and  its  usefulness  for  human-computer  interface  design 
has  not  been  systematically  investigated.    The  distinction  of  the  realistic,  analogical, 
and  logical  functions,  and  the  separation  of  pictures  into  elements,  patterns,  and  order 
of  connection  provides  a  coherent  structure  for  investigating  systematic  differences  in 
the  interpretation  of  PSSs.    Knowlton's  taxonomy  has  been  used  in  this  review  to 
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analyze  and  explain  apparently  contradictory  findings  in  recent  studies  in  this  area. 

As  a  result,  this  study  was  conducted  to  investigate  the  relationship  of  the  functional 
type  of  PSS,  viewer's  prior  experience,  and  viewing  context  to  the  ability  of 
individuals  to  interpret  PSSs  used  in  a  human-computer  interface.   Level  of 
experience  and  context  were  assumed  to  have  a  positive  effect  on  the  ability  of 
individuals  to  interpret  PSSs. 

To  test  these  assumptions  and  determine  the  relationships  stated  above,  the 
following  questions  were  raised.    Can  messages  encoded  as  realistic,  analogical,  and 
logical  pictures  be  interpreted  equally  well?  Does  an  individual's  level  of  computer- 
related  experience  affect  the  ability  to  interpret  PSSs?   Does  the  context  within  which 
a  PSS  appears  affect  interpretation?   Are  there  interactions  between  varied  levels  of 
experience,  context,  and  the  functional  type  of  PSS  that  affect  interpretation?  The 
next  chapter  will  describe  the  methodology,  design,  and  instrumentation  used  to 
answer  these  questions. 


CHAPTER  3 
METHODOLOGY 


This  study  was  designed  to  determine  whether  there  are  differences  between 
people  with  different  types  of  computer-related  experience  in  their  abilities  to  interpret 
pictorial  sign  symbols  (PSSs)  representing  features  of  a  computer  system  due  to  the 
functional  type  of  picture  used  as  a  sign  symbol.    In  addition,  the  study  explored 
whether  there  are  any  differences  in  interpretation  due  to  general  and  specific  contexts 
within  which  interpretations  are  made,  when  there  are  no  textual  cues  to  the  meanings 
of  the  visual  symbols.    Sets  of  PSSs  representing  common  word  processing  menu 
commands  were  presented  to  groups  of  teacher  education  students,  who  interpreted 
the  intended  messages.    This  chapter  includes  a  description  of  the  research  design  and 
the  methodology  used  to  gather  evidence  to  support  the  reliability  and  validity  of  the 
measures  used  in  the  conduct  of  the  experiment. 

Research  Design 

The  design  used  for  this  study  is  a  mixed  within-subject  or  repeated  measures 
factorial  design  (Keppel  &  Zedek,  1989)  using  a3x2x3x3  split  plot,  as  shown  in 
Table  3-1.   Factor  A  (computer  experience)  has  three  levels:  (a)  little  or  no  computer 
experience,  (b)  text-only  computer  experience,  and  (c)  graphic  interface  computer 
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experience.   Factor  B  (context)  has  two  levels:  (a)  low  context  (non-specific  verbal 

context),  and  (b)  high  context  (specific  verbal  context  of  computer  application  type 

and  menu  names  provided).   Factor  C  (sign  symbol  type)  has  three  levels:  (a)  realistic 

sign  symbols,  (b)  analogical  sign  symbols,  and  (c)  logical  sign  symbols.  Factor  D 

(sequence)  has  three  levels:  1,  2,  and  3,  representing  the  order  of  presentation. 

Each  subject  was  categorized  under  a  level  of  variable  A  (computer  experience) 

independently  of  assignment  to  the  context  condition.    Computer  experience  was  not 

manipulated  within  the  experiment  but  was  used  as  a  blocking  variable.    Each  subject 

was  randomly  assigned  to  one  level  of  the  context  variable.    Each  subject  received  all 

levels  of  variable  C  (PSS  types)  and  all  levels  of  variable  C  were  represented  at  each 

level  of  variable  D.   Therefore,  computer  experience  type  and  context  condition  were 

between  subjects  factors,  while  PSS  type  and  sequence  represented  within  subjects 

factors. 

Population  and  Sample 

The  population  for  this  study  was  defined  as  undergraduate  teacher  education 
students.   The  sample  for  this  study  was  recruited  from  students  enrolled  in 
introductory  educational  media  courses  at  a  college  of  education  in  a  four  year 
university.    This  population  was  of  interest  for  two  reasons:  (a)  computer  competence 
is  now  considered  a  basic  skill,  and  (b)  many  teachers  and  computer  coordinators  in 
the  public  school  system  have  little  training  in  computer  studies  and  have  mediocre 
computer  skills,  according  to  nationally  collected  data  (Martinez  &  Mead,  1988). 
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Due  to  curricular  adjustments  made  in  recent  years,  current  teacher  education  students 

may  have  higher  levels  of  knowledge  about  computers  and  greater  educational 

computing  experience  than  teachers  already  in  the  field.    One  of  the  purposes  of  this 

study  was  to  determine  whether  these  experiences  are  related  to  the  ability  to  interpret 

PSSs  similar  to  those  used  in  graphic  human-computer  interfaces. 

Students  were  recruited  from  four  intact  sections  of  introductory  educational 

media  courses  and  randomly  assigned  to  two  context  conditions.    Of  122  students 

enrolled  in  the  sections  sampled,  108  attended  on  the  days  the  study  was  conducted, 

and  all  present  participated.   The  sample,  therefore,  represented  89%  of  the 

population  at  this  location.    Due  to  the  nature  of  the  study,  written  informed  consent 

was  not  required.    Participants  were  advised  of  the  right  to  refuse  to  answer  any  of 

the  questions  on  the  survey  form,  and  several  chose  to  omit  age,  sex,  and  ethnic 

information.   Based  on  those  providing  information,  89%  were  female  and  11%  were 

males,  ranging  from  19  to  42  years  old,  with  an  average  age  of  22  years.   The  sample 

group  responses  to  ethnic  origin  categories  were  91%  White,  3.7%  Black,  and  3.7% 

Hispanic.   The  remainder  listed  Other  or  omitted  ethnic  origin.   The  sample  group 

had  completed  an  average  of  14.47  years  of  schooling.  Over  half  (53.8%)  reported 

having  obtained  an  AA  or  AS  degree.   The  number  of  computer-related  courses 

reported  ranged  from  none  to  three  courses,  with  an  average  of  1.33.   Those 

reporting  having  had  none,  one,  two,  and  three  computer-related  courses  were  17.6%, 

43.5%,  27.8%,  and  11.1%. 
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Table  3-1   Research   Design 


Within-Subjects  Factors 

Between-Subjects  Factors 

Little  or  No 

Computer 

Experience 

Text-Oriented 

Computer 

Experience 

Only 

Graphic 

Interface 

Computer 

Experience 

Context 

Context 

Context 

Sequence 

Sign  Type 

low 

high 

low 

high 

low 

high 

1 

Realistic 

n  =  12 

n  =  10 

n=9 

n=6 

n  =  33 

n=38 

Analogical 

n  =  12 

n=10 

n=9 

n=6 

n=33 

n=38 

Logical 

n=12 

n  =  10 

n=9 

n  =  6 

n=33 

n=38 

2 

Realistic 

n  =  12 

n  =  10 

n-9 

n=6 

n  =  33 

n  =  38 

Analogical 

n=12 

n  =  10 

n-9 

n=6 

n=33 

n=38 

Logical 

n  =  12 

n=10 

n=9 

n=6 

n=33 

n  =  38 

3 

Realistic 

n  =  12 

n  =  10 

n=9 

n  =  6 

n  =  33 

n=38 

Analogical 

n  =  12 

n  =  10 

n=9 

n=6 

n=33 

n=38 

Logical 

n  =  12 

n  =  10 

n=9 

n=6 

n=33 

n=38 

Materials  and  Measures 


The  PSSs  and  the  scoring  procedure  used  were  developed  locally  for  this 
study.    The  validity  of  theoretical  constructs  and  materials  is  a  primary  concern. 
Reliable  instrumentation  and  measurements  are  essential  to  meaningful  interpretation 
of  data.   Therefore,  this  section  includes  a  comprehensive  discussion  of  the  process  of 
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materials  and  instrumentation  development,  beginning  with  the  PSSs  used  before 

proceeding  to  evidence  gathered  in  support  of  the  reliability  of  the  measurements. 

This  section  describes  the  process  of  materials  development  and  the  steps  taken 
to  ensure  the  content  relevance  and  construct  validity  of  the  PSSs  generated  for  use  in 
the  study.    Messick  (1989)  has  pointed  out  the  need  for  systematic  attempts  to 
document  the  consensus  of  multiple  judges  to  deal  with  irrelevant  variance  during  test 
development  stage.    Among  the  actions  recommended  are  the  use  of  content 
specialists'  (a)  judgements  of  whether  or  not  each  item  reflects  the  content  defined  by 
the  domain,  (b)  ratings  of  items  for  the  extent  to  which  it  reflects  the  domain  facet  it 
was  intended  to  measure,  and  (c)  matching  of  items  to  the  domains  which  the  items 
best  represent.    Each  of  these  methods  was  employed  to  ensure  the  content 
representativeness  and  construct  validity  of  the  PSSs  used. 

First,  the  primary  investigator  examined  word  processing  software  menus  and 
identified  common  word  processing  menu  commands,  features,  and  options.   The 
application  software  reviewed  included  (a)  WordPerfect,  Word  and  Edit  for  the  DOS 
operating  system,  (b)  WordPerfect,  Word,  and  Write  for  the  Windows  graphical  user 
interface,  (c)  Word  and  MacWrite  for  the  Macintosh  operating  system,  and  (d) 
Pagemaker,  a  page  layout  program  with  highly  similar  Windows  and  Macintosh 
system  versions.    The  Edit  and  Write  software  applications  represent  basic  word 
processors  with  simple  features.    WordPerfect,  Word,  and  MacWrite  are  full-featured 
application  programs  that  together  represent  the  most  widely  used  word  processing 
software  applications  on  the  DOS,  Windows,  and  Macintosh  system  platforms. 
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The  investigator  selected  a  set  of  21  of  the  most  common  menu  and  command 

concepts  from  the  software  applications  surveyed.   These  options  were  available  on 

basic  and  full-featured  software  alike  and  were  represented  by  identical  or  highly 

similar  wording  on  all  programs.    For  example,  "File  Open"  and  "File  Save" 

commands  are  represented  by  the  same  words  in  all  programs.    The  menu  commands 

"Align,"  "Alignment,"  "Justify,"  and  "Justification"  represent  a  single  concept  with 

different,  but  similar  titles. 

Once  the  pool  of  command  concepts  had  been  identified,  three  PSSs 
constructed  to  meet  the  definitions  of  realistic,  analogical,  and  logical  pictures 
according  to  Knowlton's  taxonomy  were  drawn  for  each  command.    An  effort  was 
made  to  avoid  exact  copies  of  icons  used  in  currently  available  software,  but  the  use 
of  similar  representations  was  not  ruled  out.    A  public  domain  image  editing  program 
for  the  Macintosh  was  used  to  generate  and  save  the  pictures  in  machine  readable  file 
formats.   The  pictures  were  each  drawn  within  a  one-inch  frame  and  displayed  on  the 
computer  video  display  screen  prior  to  printing. 

Expert  review  of  representation  of  taxonomy  categories.    The  63  pictures  were 
printed  and  copied  to  card  stock  paper  and  cut  into  one-inch  squares.   The  paper 
squares  were  then  shuffled  and  mixed  before  presentation  to  two  educational  media 
experts  familiar  with  Knowlton's  taxonomy,  who  independently  sorted  the  pictures 
into  the  three  categories.    Full  agreement  on  the  functional  types  of  the  pictures  was 
attained  following  discussion  of  picture  referents  and  interpretations.    The  expert 
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reviewers  provided  comments  and  suggestions  regarding  the  interpretation  and  design 

of  the  pictures. 

Expert  matching  of  sign  symbols  to  domain  content  represented.    The  set  of  21 
menu  command  concepts  and  the  21  groups  of  three  PSSs  were  printed  and  pasted  on 
separate  sets  of  4"  x  6"  index  cards.    Both  groups  of  21  cards  were  shuffled  and 
presented  to  each  of  two  computer  usage  experts.    Both  subjects  achieved  100% 
accuracy  in  matching  the  sets  of  three  PSSs  with  the  corresponding  written  menu 
commands. 

The  two  educational  media  experts  were  then  provided  with  the  individual  sets 
of  21  PSSs  by  functional  type.    Each  set  of  pictures  was  randomized.   The  assigned 
task  was  to  match  each  individual  PSS  with  the  intended  menu  and  command.   The 
sequence  of  presentation  of  each  full  set  of  pictures  was  randomized  and  cycled.    The 
experts  achieved  more  than  80%  success  in  matching  the  PSSs  to  the  intended  menus 
and  commands.    Misinterpretations  and  crossed  interpretations  of  the  intended 
messages  were  noted  and  used  in  the  decision  process  to  reduce  the  pictures  to  the 
experimental  set.    For  example,  the  PSSs  for  "File  New"  and  "File  Open"  were 
misinterpreted  in  some  instances.    The  "File  New"  command  was  eventually 
eliminated  from  the  sets  used  in  the  study,  and  both  "New"  and  "Open"  were 
accepted  as  correct  interpretations  of  the  intended  message  of  the  "File  Open" 
symbol. 

Expert  ratings  of  sign  symbols  regarding  domain  content  representation. 
Finally,  each  group  of  21  sign  symbols  was  printed  on  a  sheet  of  letter  size  paper  and 
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organized  in  a  menu  sequence  consistent  with  that  found  in  the  software  applications 

reviewed.   A  panel  of  six  experts  evaluated  how  well  each  of  the  PSSs  communicated 

or  represented  the  intended  menu  command,  using  a  five-point  Likert  scale.   These 

results  are  shown  in  Table  3-2.    Using  these  ratings,  the  three  highest  ranked  symbols 

from  each  of  four  groups  of  related  concepts  were  selected  for  inclusion  in  the 

experimental  set  of  36  pictures  representing  12  concepts.  The  36  PSSs  were  randomly 

assigned  to  three  groups  so  each  group  contained  an  equal  number  of  realistic, 

analogical,  and  logical  pictures.    Overhead  transparencies  were  constructed  so  each 

symbol  was  equal  in  size.    Each  overhead  transparency  consisted  of  three  PSSs 

representing  commands  typically  found  on  the  same  menu.  Four  overhead 

transparencies  were  used  in  each  set,  representing  four  different  menus. 

Procedure 

The  experimental  setting  was  the  classroom  within  which  the  classes  normally 
met.   The  classroom  contains  student  desks,  tables,  chalkboards,  and  a  video 
projection  screen.    All  groups  viewed  the  materials  under  a  standard  presentation 
protocol.    Experimental  materials  were  projected  against  a  wall  designed  for  and 
normally  used  for  projected  visuals,  so  the  materials  were  readily  visible  to  all 
subjects.    Materials  were  presented  to  subjects  in  intact  sections.    Subjects  within  each 
section  were  randomly  assigned  to  either  a  low  context  or  a  high  context  condition. 
The  context  condition  was  manipulated  experimentally  through  random  assignment  of 
differential  instructions  to  subjects. 
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Table  3-2  Expert  Ratings  of  Pictorial  Sign  Symbols 


Variable 

H 

Mean 

SP 

SJa 

File  New 

18 

3.28 

1.18 

.28 

File  Open 

18 

3.33 

1.28 

.30 

File  Save 

18 

3.78 

.88 

.21 

File  Print 

18 

4.00 

.97 

.23 

File  Exit 

18 

3.17 

1.10 

.26 

Edit  Undo 

18 

3.44 

1.15 

.27 

Edit  Cut 

18 

4.22 

.88 

.21 

Edit  Copy 

18 

4.11 

.83 

.20 

Edit  Paste 

18 

3.5 

1.20 

.28 

Justify  Left 

18 

4.72 

.46 

.11 

Justify  Center 

18 

4.56 

.62 

.15 

Justify  Right 

18 

4.72 

.46 

.11 

Justify  Full 

18 

4.44 

.78 

.18 

Single  Space 

18 

4.39 

.85 

.20 

1.5  Line  Space 

18 

4.44 

.86 

.20 

Double  Space 

18 

4.44 

.86 

.20 

Prior  to  presentation  of  the  PSSs,  verbal  informed  consent  procedures  were 
followed  and  subjects  were  requested  to  complete  a  locally  developed  instrument 
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assessing  computer  experience  and  a  self-assessment  of  experience  and  expertise. 

Following  presentation  of  the  stimuli  and  collection  of  the  data,  each  subject  was 

coded  to  one  of  three  computer  experience  categories.    The  categories  of  experience 

were  (a)  novice,  (b)  text-only,  and  (c)  graphic  interface  experience. 

Subjects  assigned  to  the  low  context  condition  were  given  the  following 
explanation:  "You  will  be  shown  some  pictures,  commonly  called  "icons,"  which  are 
intended  to  represent  commands,  features,  or  functions  of  a  microcomputer  system. 
These  pictures  will  be  shown  in  groups.    Each  group  of  pictures  will  be  shown  for  the 
same  amount  of  time.   Indicate  whether  you  recognize  an  icon  from  having  seen  or 
used  it  in  a  software  program  before  by  checking  either  Y  for  "yes"  or  N  for  "no." 
Describe  or  give  a  name  for  each  numbered  picture  and  describe  the  computer  feature 
that  you  think  it  stands  for.    Please  be  as  specific  as  possible." 

Subjects  assigned  to  the  high  context  condition  were  given  the  following 
explanation:  "You  will  be  shown  some  pictures  which  are  intended  to  represent 
commands,  features,  or  functions  of  a  microcomputer  system.   These  pictures  will  be 
shown  in  groups.   Each  group  of  pictures  will  be  shown  for  the  same  amount  of  time. 
Each  individual  picture  stands  for  a  feature  that  can  be  found  within  a  word 
processing  program.    Imagine  that  you  have  been  given  a  marked  up,  edited  version 
of  a  term  paper  that  you  need  to  rewrite.    The  word  processor  commands  and  features 
are  indicated  only  by  these  pictures;  there  are  no  written  labels.    The  groups  of 
pictures  you  see  are  the  word  processor  menu  options  (File,  Edit,  Format)  available 
to  you  on  the  computer  screen.    Each  numbered  picture  is  intended  to  convey  to  you 
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the  name  or  function  of  the  feature  it  represents.    Each  picture  will  be  shown  for  the 

same  amount  of  time.    Indicate  whether  you  recognize  an  icon  from  having  seen  or 

used  it  in  a  software  program  before  by  checking  either  Y  for  "yes"  or  N  for  "no." 

Describe  or  name  each  picture  and  describe  the  computer  command,  function,  or 

feature  that  you  think  it  stands  for.   Please  be  as  specific  as  possible." 

PSSs  were  presented  to  each  section  following  the  same  procedure.    All 
subjects  viewed  PSSs  within  each  of  three  sign  symbol  types:  realistic,  analogical,  and 
logical  pictures.   These  symbols  were  randomly  assigned  to  three  groups  of  symbol 
sets  so  each  process  or  function  was  represented  within  each  group.    Subjects  viewed 
one  of  three  groups  of  sign  symbols  at  each  measurement.    The  order  of  viewing  the 
symbol  sets  was  cycled  to  defend  against  the  possibility  of  treatment  interactions  due 
to  the  sequence  of  presentation.    Since  the  viewer  should  be  able  to  infer  an  icon's 
meaning  on  the  first  encounter  (Lodding,  1983),  all  three  sets  of  pictures  were 
presented  in  one  session  of  less  than  one  hour.   The  symbols  were  presented  in  a 
standard  fashion,  so  each  symbol  was  viewed  for  the  same  amount  of  time.   Three 
symbols,  numbered  from  one  to  three,  were  presented  simultaneously  for  one  minute 
on  each  overhead  transparency. 

The  instrument  used  for  recording  subjects'  responses  had  two  forms.   The 
forms  differed  for  high  context  and  low  context  conditions  solely  in  the  written 
instructions  provided  to  the  subject.    The  main  portion  of  the  form  consisted  of  a 
numbered  table.    Each  numbered  row  was  used  to  respond  to  an  individual  PSS. 
Subjects  were  asked  to  identify,  by  checking  a  box,  symbols  that  were  recognized 
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from  prior  experience.   For  all  symbols,  subjects  were  asked  to  provide  a  name  and  a 

brief  description  of  the  function  or  feature  to  which  the  symbol  referred. 

Scoring  was  made  on  the  basis  of  the  number  of  correct  interpretations  of  the 
PSSs  used  in  the  study.   Thus,  each  subject's  score  could  range  from  zero  to  12  per 
occasion,  and  the  total  score  for  each  could  range  from  zero  to  36.   A  subject's  score 
for  a  specific  set  of  PSSs  also  could  range  from  zero  to  12. 

Reliability  of  scoring.    A  pilot  study  was  conducted  using  18  graduate  students 
enrolled  in  a  course  in  educational  television  design  and  production  to  test  the 
presentation  protocol  and  evaluate  the  reliability  of  the  scoring  protocol.    The 
assessment  measures  and  protocol  are  described  below. 

The  investigator  developed  a  survey  of  computer  experience  following  a 
review  of  currently  available  computer  literacy  instruments  and  the  nature  of 
Knowlton's  taxonomy.   The  survey  of  computer  experience  asked  subjects  to  check 
off  hardware  and  software  items  with  which  they  have  had  experience.    The  items 
were  grouped  into  IBM  PC  compatible  (text  based)  interfaces,  Windows  and 
Macintosh  (graphics)  interfaces,  and  other  hardware  and  software.    The  pattern  of 
items  checked  was  used  by  judges  following  a  written  protocol  to  assign  each  subject 
to  one  of  the  three  computer  experience  categories.    In  addition,  a  five-item  self- 
assessment  survey  was  created  using  a  five  point  scale.    The  first  two  items  asked 
subjects  to  indicate  how  often  and  for  how  long  they  have  used  computers.    The  last 
three  items  asked  subjects  to  assess  themselves,  from  novice  through  expert  levels, 
regarding  overall  computer  experience  and  experience  related  to  text  and  graphic 
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interfaces.    The  judges  used  this  assessment  to  assign  each  subject  to  one  of  the  three 

computer  experience  categories.    Finally,  a  composite  category  was  assigned  based  on 

the  categories  to  which  each  subject  had  been  assigned  for  the  two  separate 

instruments. 

Response  scoring  was  conducted  independently  of  categorization  of  experience 
type.   Each  judge  was  provided  with  an  answer  key  providing  the  intended  menus  and 
commands  in  the  order  presented.    Incorrect  responses  and  responses  that  indicated 
the  subject  was  unable  to  guess  a  meaning  were  scored  as  zero  points.    Each  correct 
response  was  scored  as  one  point.    An  answer  was  considered  correct  if  the  command 
name  or  brief  description  matched  or  was  highly  similar  to  that  on  the  answer  key. 
Each  partially  correct  response  was  scored  as  one-half  point.    An  answer  was 
considered  partially  correct  if  the  subject  correctly  identified  the  intended  menu  but 
did  not  specify  the  command  function.    For  example,  if  the  "File  Open"  PSS  had 
been  displayed  and  the  response  had  been  "File,"  the  answer  would  be  scored  as 
partially  correct. 

For  the  pilot  project,  the  judges  reached  a  consensus  of  94%  in  categorizing 
subjects  to  computer  experience  categories.    Correlation  of  judges'  scoring  of  subjects 
in  the  pilot  project  was  high,  with  a  coefficient  alpha  of  .995  reported. 

Due  to  the  high  level  of  agreement  on  the  pilot  project  for  both  the 
categorization  and  the  scoring  instruments,  one  judge  performed  the  categorization 
and  scoring  of  the  study  data.    A  random  sampling  of  20  subjects  was  independently 
categorized  and  scored  by  each  of  the  other  two  judges  used  for  the  pilot  study. 
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Written  instructions  were  provided  for  the  categorization  and  scoring  tasks,  with  no 

retraining  of  judges.   The  level  of  consensus  for  the  categorization  task  was  .85  and  a 

coefficient  alpha  of  .977  was  calculated  for  the  scoring  task.    The  lower  level  of 

consensus  regarding  the  categorization  of  experience  was  expected,  as  some  of  the 

participants  in  the  study  were  enrolled  in  an  introductory  course  in  instructional 

computing.   The  concurrent  exposure  in  class  and  laboratory  to  the  Macintosh 

graphical  user  interface  may  have  influenced  subjects'  responses  to  the  computer 

experience  survey  and  the  self-assessment  instrument.   The  graduate  students  sampled 

in  the  pilot  study  were  more  clearly  experienced  or  not  experienced  with  the  different 

types  of  user  interfaces.   This  evidence  provides  additional  support  for  the  reliability 

of  the  instruments  and  instructions  developed  for  categorization  of  subjects  and 

scoring  of  subjects'  responses. 

The  following  null  hypotheses  were  derived  from  the  research  design  for 
testing  using  a  four-way  analysis  of  variance  with  repeated  measures  statistical 
procedure. 

Hypothesis  1.    There  will  be  no  significant  differences  within  subjects  in  the 
ability  to  interpret  different  types  of  PSSs. 

Hypothesis  2.    There  will  be  no  significant  differences  in  the  ability  to 
interpret  PSSs  between  subjects  viewing  PSSs  within  a  given  context  and  subjects 
viewing  PSSs  with  no  specific  context. 

Hypothesis  3.    There  will  be  no  significant  differences  in  the  ability  to 
interpret  PSSs  between  subjects  with  different  levels  of  computer  experience. 
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Hypothesis  4.   There  will  be  no  significant  differences  in  the  ability  to 

interpret  PSSs  within  subjects  due  to  the  sequence  in  which  PSSs  are  viewed. 

Hypothesis  5.    There  will  be  no  significant  differences  in  the  ability  to 

interpret  PSSs  due  to  interactions  among  the  functional  type  of  PSS  viewed,  the  type 

of  computer  experience,  the  viewing  context  conditions,  and  the  sequence  of  viewing. 

Summary 

The  use  of  PSSs  within  the  human-computer  interface  has  created  a  need  for 
comprehensive  guidelines  for  the  construction  of  these  non-verbal  messages.   This 
study  was  conducted  to  provide  data  relating  PSS  functional  types  to  the  ability  of 
individuals  to  interpret  the  intended  messages.    Recent  research  results  were 
incorporated  into  the  design  of  the  study  to  determine  whether  there  are  differences  in 
the  ability  to  interpret  PSSs  due  to  differences  in  functional  type  based  on  Knowlton's 
taxonomy.   This  study  also  was  designed  to  determine  whether  differences  in  context 
and  computer  related  experience  affect  the  interpretation  of  PSSs  and  whether  there 
are  any  interactions  among  these  variables.    Information  to  be  gained  from  this  study 
has  implications  for  and  could  be  useful  to  designers  of  computer  systems,  designers 
of  computer-based  instructional  systems,  designers  of  educational  media,  and 
educators  using  computers  as  tools  to  improve  learning. 

To  produce  interpretable  results,  the  investigator  relied  upon  evidence  gathered 
to  support  the  content  representativeness  and  construct  validity  of  the  materials  used 
as  experimental  stimuli.   The  investigator  developed  instruments  for  categorizing 
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subjects  by  computer  experience  type  and  for  scoring  interpretation  responses.    A 

limited  pilot  project  was  conducted  to  assess  the  reliability  of  these  instruments  across 

multiple  judges.   The  research  design  used  an  analysis  of  variance  with  two  between- 

subjects  and  two  within-subjects  factors  to  take  advantage  of  the  increased  power  of 

repeated  measures.    Steps  were  taken  in  the  implementation  of  the  research  design  to 

defend  against  the  possibility  of  differential  carryover  or  practice  effects.    The  study 

was  conducted  and  the  resulting  data  are  reported  and  analyzed  in  the  next  chapter. 


CHAPTER  4 
RESULTS  AND  ANALYSIS 


Introduction 


This  study  was  designed  to  determine  whether  the  functional  type  of  pictorial 
sign  symbol,  as  defined  by  James  Q.  Knowlton's  taxonomy  (Knowlton,  1964,  1966), 
affects  the  interpretation  of  pictorial  sign  symbols.    Computer  experience  was  used  as 
a  between-subjects  attribute  variable  with  three  levels  (novice,  text-only,  and 
graphics).    Viewing  context  was  manipulated  as  a  between-subjects  experimental 
condition  with  two  levels  (low  context  and  high  context).    Pictorial  sign  symbol  type 
was  used  as  a  within-subjects  factor  with  three  levels  (realistic,  analogical,  and  logical 
pictorial  sign  symbols).    Sequence  was  used  as  a  within-subjects  factor  with  three 
levels.    A3x2x3x3  ANOVA  design  was  used  for  the  study.    Statistical  analyses 
of  the  data  were  conducted  using  the  General  Linear  Models  (GLM)  procedure  of  the 
SAS  statistical  analysis  software  program.    The  results  of  this  study  are  reported 
below.   Statistically  significant  effects  related  to  pictorial  sign  symbol  type,  computer 
experience,  sequence  of  viewing,  and  the  interaction  between  pictorial  sign  symbol 
type  and  sequence  were  found.    Implications  of  these  results  are  discussed  in  the  final 
chapter. 
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Results 

The  experiment  was  conducted  and  data  were  collected  as  described  in  the 
previous  chapter.   The  data  were  analyzed  using  a  four-way  ANOVA  model  including 
repeated  measures  on  the  sign  symbol  type  factor.    These  results  are  presented  as 
source  tables  in  Table  4-1  and  Table  4-2  and  are  discussed  separately  below.   The 
report  of  results  begins  with  a  review  of  the  null  hypotheses  tested. 
Table  4-1  Analysis  of  Variance  Summary  Table— Between  Subjects  Effects 
Source  df        SS        MS       F  Pr>F 


Experience  2  9.61  34.31  12.00  .0001 

Context  1  2.91  2.91  1.00  .3188 

Experience  x  Context  2  1.33  .37  0.23  .7951 

Error  102  295.95  2.90 


Hypothesis  1  stated  that  there  would  be  no  significant  differences  in  the  ability 
to  interpret  different  types  of  pictorial  sign  symbols.    The  analysis  of  variance 
procedure  produced  an  F  value  of  52.28  (df  2,  204)  that  was  statistically  significant  at 
the  .05  alpha  level  (see  Table  4-2).   Therefore,  the  null  hypothesis  was  rejected.   The 
functional  type  of  sign  symbol  viewed  was  related  to  the  ability  to  interpret  the 
symbol. 

Hypothesis  2  stated  that  there  would  be  no  significant  differences  between 
subjects  viewing  pictorial  sign  symbols  within  a  high  context  condition  and  subjects 


Table  4-2  Analysis  of  Variance  Summary  Table— Within  Subjects  Effects 


Source 


df 


SS         MS 


Pr>F 
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Sequence 

Sequence  x  Experience 
Sequence  x  Context 
Error 


4.96     2.48     4.53       .0119 


3.50       .88     1.60       .1763 


.23 


11      1.97       .1003 


204       111.88       .55 


Sign 

Sign  X  Experience 

Sign  X  Context 

Error 


57.79  28.90  52.28       .0001 


3.90       .97     1.76       .1375 


.34       .17       .31        .7345 


204       112.75        .55 


Sequence  x  Sign 

4 

6.33 

1.58 

4.03 

.0032 

Sequence  x  Sign  x  Experience 

8 

2.40 

.30 

.76 

.6353 

Sequence  x  Sign  x  Context 

4 

.40 

.10 

.26 

.9053 

Sequence  x  Sign  x  Experience  x 

Context  8 

3.45 

.43 

1.10 

.3623 

Error 

408 

160.11 

.39 

viewing  pictorial  sign  symbols  within  a  low  context  condition  in  the  interpretation  of 
the  pictorial  sign  symbols.  The  statistical  analysis  produced  an  F  ratio  of  1.00  (df  1, 
102).   This  value  is  not  significant  at  the  .05  alpha  level  (see  Table  4-1).   The  null 
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hypothesis  was  not  rejected.    Context,  as  operationalized  for  this  study,  was  not 

significantly  related  to  the  interpretation  of  pictorial  sign  symbols. 

Hypothesis  3  stated  that  there  would  be  no  significant  differences  in  the  ability 
to  interpret  between  subjects  with  different  types  of  computer  experience.    The 
analysis  produced  an  F  value  of  12.00  (df  2,  102)  that  was  significant  at  the  .05  alpha 
level  (see  Table  4-1).   The  null  hypothesis  was  rejected.   The  type  of  computer 
related  experience,  as  categorized  for  this  study,  was  related  to  the  ability  of  subjects 
to  interpret  the  pictorial  sign  symbols  used  in  this  study. 

Hypothesis  4  stated  that  there  would  be  no  significant  differences  within 
subjects  in  the  ability  to  interpret  associated  with  the  sequence  in  which  pictorial  sign 
symbols  were  viewed.    The  observed  F  value  of  4.53  (df  2,  204)  was  statistically 
significant  at  the  .05  alpha  level  (see  Table  4-2).    The  null  hypothesis  was  rejected. 
The  ability  to  interpret  pictorial  sign  symbols  was  related  to  the  sequence  in  which  the 
subjects  viewed  the  groups  of  symbols. 

Hypothesis  5  stated  that  there  would  be  no  significant  differences  between 
subjects  with  differing  types  of  experience  viewing  pictorial  sign  symbols  under 
differing  context  conditions  and  in  different  orders  of  viewing  in  the  ability  to 
interpret  different  functional  types  of  pictorial  sign  symbols.    Stated  more  simply,  this 
hypothesis  stated  that  there  would  be  no  interactions  between  the  variables  measured. 
For  the  Sequence  x  Pictorial  Sign  Symbol  type  interaction,  the  observed  F  value  of 
4.03  (df  4,  408)  was  statistically  significant  at  the  .05  alpha  level  (see  Table  4-2). 
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The  null  hypothesis  was  rejected.   The  ability  to  interpret  different  functional  types  of 

pictorial  sign  symbols  was  related  to  the  sequence  in  which  the  symbols  were  viewed. 

Although  the  tests  of  hypotheses  resulted  in  significant  main  effects  for 

pictorial  sign  symbol  type,  sequence  of  viewing,  and  computer  experience,  these 

results  are  of  secondary  interest  to  the  effect  of  the  observed  interaction.    The 

following  section  will  review  the  salient  aspects  of  the  analysis  of  these  data. 

Analysis 

Main  effects  were  found  for  pictorial  sign  symbol  type,  computer  experience 
type,  and  sequence  of  viewing  variables.    An  interaction  between  the  pictorial  sign 
symbol  and  sequence  factors  was  observed.    For  all  analyses,  the  Greenhouse-Geisser 
Epsilon  statistic  was  non-significant,  indicating  that  the  assumption  of  homogeneity  of 
variances  was  not  violated.   For  all  significant  effects,  the  G-G  adjustment  did  not 
affect  the  statistical  significance  of  the  observed  F  value. 

To  determine  the  cell  comparisons  of  interest,  statistics  were  calculated  and 
cell  means  were  plotted.   Means  and  standard  deviations  for  the  pictorial  sign  symbol 
types  at  the  different  sequence  levels  are  included  in  Table  4-3,  and  these  means  are 
plotted  in  Figure  4-1.    Single  degree  of  freedom  contrasts  were  used  to  compare  the 
cell  means  across  adjacent  sequence  positions  within  each  sign  symbol  type. 

Significant  differences  were  found  between  sequence  1  and  sequence  2,  and 
between  sequence  2  and  sequence  3  for  realistic  pictorial  sign  symbols  (see 
Table  4-4).   There  were  no  significant  differences  by  sequence  between  means  for 


Table  4-3   Means  of  Pictorial  Sign  Symbols  by  Sequence  of  Viewing 

Variable  N  M         Std  Dey  Std  Err 

Sequence  Level  One 

Realistic  108            .60               .65  .06 

Analogical  108  1.36  1.03  .10 

Logical  108            .32               .50  .05 

Sequence  Level  Two 

Realistic  108            .79               .83  .08 

Analogical  108  1.20  1.17  .11 

Logical  108            .59               .71  .07 

Sequence  Level  Three 

Realistic  108  1.13  1.06  .10 

Analogical  108  1.43  1.14  .11 

Logical  108            .59               .75  .07 
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analogical  pictorial  sign  symbols  (see  Table  4-5).   A  significant  difference  was  found 
between  sequence  1  and  sequence  2,  but  not  between  sequence  2  and  sequence  3,  for 
logical  pictorial  sign  symbols  (see  Table  4-6).    Main  effects  were  found  for  the 
pictorial  sign  symbols  type,  sequence  of  viewing,  and  type  of  computer  experience 
variables. 
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2.00 


Realistic 


Sequence 

-  Analogical  — •—  Logical 


Figure  4-1    Means  of  Pictorial  Sign  Symbols  by  Sequence  of  Viewing 

Due  to  the  more  complex  interaction  of  the  sign  symbols  type  and  sequence 
variables,  tables  for  the  main  effects  of  sign  symbol  type  and  sequence  are  not 
presented  here.    From  a  possible  range  from  zero  to  12  correct  interpretations  for 
each,  overall  means  for  logical,  realistic,  and  analogical  PSSs  were  1.51,  2.51,  and 
3.99  respectively.    The  contrast  comparisons  between  the  PSS  types  were  all 
statistically  significant  (p  >  .0001),  indicating  that  each  of  the  functional  types  of 
pictures  within  the  Knowlton  taxonomy  was  different  from  the  others. 

The  means  for  the  first,  second,  and  third  viewing  sequence  levels  were  2.29, 
2.58,  and  3.14.    The  contrast  comparisons  between  adjacent  groups  were  not 
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Table  4-4  Repeated  Measures  Analysis  of  Variance 
Contrast  Between  Realistic  Sign  Symbols  and  Sequence 


Source 


df 


SS 


MS 


Pr>F 


Contrast  of  Sequence  One  vs  Sequence  Two 


Mean 

1 

3.42 

3.42 

5.40 

.0221 

Experience  Type 

2 

.33 

.17 

.26 

.7686 

Context 

1 

.52 

.52 

.82 

.3662 

Experience  Type  x 

Context 

2 

1.64 

.82 

1.30 

.2769 

Error 

102 

64.48 

.63 

Contrast  of  Sequence  Two  vs  Sequence  Three 
Mean  1 


4.06     4.06     6.10    .0152 


Experience  Type 

2 

2.84 

1.42 

2.13 

.1239 

Context 

1 

.03 

.03 

.04 

.8360 

Experience  Type  x  Context 

2 

1.90 

.95 

1.42 

.2453 

Error 

102 

67.90 

.67 

statistically  significant,  but  the  comparison  between  the  first  and  third  sequence 
positions  was  statistically  significant  (p  >  .0068),  indicating  that  subjects'  ability  to 
interpret  the  PSSs  was  greater  for  the  third  group  viewed  than  for  the  initial  exposure 
group.   These  main  effects  for  sequence  and  sign  symbol  type  variables  should  be 
considered  in  the  context  of  the  Sequence  x  Pictorial  Sign  Symbol  interaction. 
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Table  4-5  Repeated  Measures  Analysis  of  Variance 
Contrast  Between  Analogical  Sign  Symbols  and  Sequence 


Source 


df 


SS 


MS 


Pr>F 


Contrast  of  Sequence  One  vs  Sequence  Two 
Mean  1 

Experience  Type  2 

Context  1 

Experience  Type  x  Context  2 

Error 


1.83      1.83      1.08    .3020 


.03       .01 


.01    .9915 


.17 


,17       .10    .7518 


6.75      3.38      1.99    .1421 


102       173.27      1.70 


Contrast  of  Sequence  Two  vs  Sequence  Three 

Mean  1  .51 


Experience  Type 

Context 

Experience  Type  x  Context 

Error 


.51       .38    .5390 


5.62      2.81      2.10    .1278 


.31 


.31        .23    .6312 


3.62      1.81      1.35    .2637 


102       136.60      1.34 


In  addition  to  the  effects  of  sign  symbol  type  and  sequence,  a  main  effect  due 
to  the  type  of  previous  computer  experience  was  observed.    This  effect  was  consistent 
across  sequence  and  pictorial  sign  symbol  type.    Statistics  are  shown  in  Table  4-7  and 
are  graphed  in  Figure  1-2.   Tukey's  HSD  statistic  was  calculated  at  an 
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Table  4-6  Repeated  Measures  Analysis  of  Variance 
Contrast  Between  Logical  Sign  Symbols  and  Sequence 


Source 


df 


SS 


MS 


Pr>F 


Contrast  of  Sequence  One  vs  Sequence  Two 
Mean  1 

Experience  Type  2 

Context  1 

Experience  Type  x  Context  2 

Error  102 


3.72      3.72     9.15    .0031 


.27 


41.41 


,13        .33    .7187 


.00        .00       .00    .9617 


.03        .01 


.41 


.03    .9690 


Contrast  of  Sequence  Two  vs  Sequence  Three 
Mean  1 

Experience  Type  2 

Context  1 

Experience  Type  x  Context  2 

Error  102 


.09        .09       .27    .6018 


.20        .10       .32    .7297 


.03        .03 


32.23 


.09    .7610 


1.08       .54      1.72    .1850 


.32 


alpha  level  of  .05,  producing  a  minimum  significant  mean  difference  of  1.18, 
exceeded  by  each  of  the  pairwise  contrasts  between  means. 

To  summarize,  the  analyses  of  data  collected  from  this  study  resulted  in  the 
following  findings.   Significant  differences  in  the  ability  of  subjects  to  interpret 
pictorial  sign  symbols  were  found  related  to  the  interaction  between  the  sign  symbol 
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Figure  4-2  Means  by  Type  of  Computer  Experience 

type  viewed  and  the  sequence  of  viewing.   Significant  differences  were  found  in  the 
ability  of  subjects  to  interpret  pictorial  sign  symbols  associated  with  the  taxonomic 
classifications  of  these  symbols  into  realistic,  analogical,  and  logical  categories. 
Significant  differences  were  found  in  the  ability  of  subjects  to  interpret  pictorial  sign 
symbols  related  to  subjects'  type  of  computer  experience  as  operationalized  for  this 
study  into  novice,  text-only,  and  graphic  interface  experience  categories.    Significant 
differences  were  found  in  the  ability  of  subjects  to  interpret  pictorial  sign  symbols 
related  to  the  sequence  in  which  the  symbols  were  viewed.   The  ability  of  subjects  to 
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Table  4-7  Means  by  Computer  Experience  Tvpe 

Category  of  Computer  Experience             n 

M  Std  Dev    Std  Err 

Novice  22  3.98     2.76       .59 

Text-Only  15  5.63      4.06      1.05 

Graphics  71  9.76     5.74       .68 


interpret  pictorial  sign  symbols  was  not  significantly  affected  by  the  viewing  context 

as  operationalized  for  this  study. 

This  study  was  designed  to  determine  whether  Knowlton's  taxonomy  of  the 

functions  of  pictures  could  be  applied  to  the  design  of  pictorial  sign  symbols  used  to 

represent  commands  in  the  human-computer  interface.    How  do  these  results  affect 

the  knowledge  base  used  by  instructional  message  designers,  human-computer 

i 
interface  designers,  and  developers  of  educational  software  programs?  These  , 

questions  are  addressed  in  the  next  chapter.  J 


CHAPTER  5 
CONCLUSIONS  AND  RECOMMENDATIONS 


Introduction 

The  purpose  of  this  study  was  to  investigate  the  relationship  of  the  functional 
type  of  picture,  as  defined  by  Knowlton's  taxonomy  (Knowlton,  1964,  1966),  to  the 
ability  of  subjects  with  different  types  of  experience  to  interpret  pictorial  sign  symbols 
(PSSs)  representing  commands  used  in  a  graphic  human-computer  interface,  under 
varying  context  conditions.   The  research  design  utilized  a3x2x3x3  factorial 
design  with  repeated  measures.   Factors  measured  included  computer  experience  type 
(three  levels),  context  (two  levels),  pictorial  sign  symbol  type  (three  levels),  and 
sequence  of  viewing  (three  levels). 

To  implement  this  study,  teacher  education  students  were  chosen  as  the 
population  of  interest  from  which  the  sample  was  taken.    Materials  were  developed  to 
represent  word  processing  software  commands  in  each  of  the  three  functional  types  of 
pictures,  using  expert  opinions  systematically   to  substantiate  the  construct  validity  of 
the  materials  used.    Instruments  for  categorizing  subjects  into  computer  experience 
categories  and  for  scoring  interpretations  were  developed  and  pilot  tested  to 
substantiate  the  reliability  of  measurements.    The  results  of  this  study  in  terms  of  the 
research  hypotheses  are  presented  next. 
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Findings 

The  study  was  implemented,  data  were  collected,  and  the  data  were  analyzed 
in  terms  of  the  stated  hypotheses.   The  research  questions  had  been  stated  by  the 
following  null  hypotheses: 

Hypothesis  I.   There  will  be  no  significant  differences  within  subjects  in  the 
ability  to  interpret  different  types  of  pictorial  sign  symbols.    This  hypothesis  was 
rejected. 

Hypothesis  2.    There  will  be  no  significant  differences  in  the  ability  to 
interpret  PSSs  between  subjects  viewing  pictorial  sign  symbols  within  high  context 
and  low  context  conditions.   This  hypothesis  was  not  rejected. 

Hypothesis  3.   There  will  be  no  significant  differences  in  the  ability  to 
interpret  PSSs  between  subjects  with  different  types  of  computer  experience.    This 
hypothesis  was  rejected. 

Hypothesis  4.    There  will  be  no  significant  differences  in  the  ability  to 
interpret  PSSs  within  subjects  related  to  the  sequence  in  which  pictorial  sign  symbols 
are  viewed.   This  hypothesis  was  rejected. 

Hypothesis  5.   There  will  be  no  significant  differences  in  the  ability  to 
interpret  PSSs  due  to  interactions  among  the  functional  type  of  PSS  viewed,  the  type 
of  computer  experience,  the  viewing  context  conditions,  and  the  sequence  of  viewing. 
This  hypothesis  was  rejected. 


■■^st:? 
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Specifically,  an  interaction  was  found  between  pictorial  sign  symbol  type  and 

the  sequence  of  viewing.   Main  effects  were  found  for  computer  experience  type,  for 

pictorial  sign  symbol  type  and  sequence  of  viewing.   The  viewing  context,  as 

operationalized  for  this  study,  did  not  result  in  a  statistically  significant  effect  on  the 

interpretation  of  sign  symbols.   These  results  are  interpreted  in  the  next  section. 

Discussion 

The  most  interesting  result  of  this  study  was  the  differential  effect  of  sequence 
on  the  interpretation  of  the  different  types  of  PSSs.    As  shown  in  Table  4-1,  each  of 
the  sign  symbol  types  displayed  a  different  pattern  of  correct  interpretations  over 
sequence  order,  creating  an  ordinal  interaction. 

Specifically,  realistic  PSSs  were  interpreted  poorly  during  the  first  sequence, 
but  each  subsequent  viewing  group  resulted  in  a  statistically  significant  positive 
difference.    Analogical  PSSs  were  interpreted  correctly  at  a  higher  level  than  other 
PSSs,  but  the  average  number  of  correct  interpretations  varied  little  and  did  not 
increase  or  decrease  significantly  over  time.    Logical  PSSs  were  initially  interpreted 
at  a  relatively  low  level  of  correct  interpretations  and  increased  significantly  between 
the  first  and  second  sequence.   However,  the  average  number  of  correct 
interpretations  remained  the  same  for  the  second  and  third  sequence  positions. 

The  interpretation  of  these  data  must  take  into  account  that  the  population  used 
for  this  study,  teacher  education  students,  was  homogenous,  consisting  primarily  of 
young  adult  white  females  with  little  computer  science  background.    The  general 
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application  of  these  results  to  other  populations  is  not  recommended.    The  nature  of 

the  task  domain  (word  processing)  also  should  be  considered  before  attempting  to 

generalize  these  results  to  other  tasks  involving  human-computer  interaction. 

A  main  effect  due  to  sequence  was  observed,  although  the  research  design  was 
structured  to  reduce  the  possibility  of  practice  effects.    This  suggests  that  the  repeated 
exposure  to  the  twelve  concepts  represented  by  the  36  PSSs  may  have  influenced  the 
decoding  of  the  concepts  and  produced  the  higher  average  of  correct  interpretations  on 
the  subsequent  exposures.    The  sequence  effect  was  significant  for  realistic  PSSs, 
indicating  that  as  more  examples  were  viewed,  subjects  were  able  to  decode  and 
interpret  the  realistic  symbols  designed  for  this  study  more  effectively. 

The  analogical  PSSs  were  interpreted  at  a  consistently  high  level.    One 
possible  explanation  may  be  that  for  those  with  graphic  interface  experience,  the  high 
level  of  interpretation  was  in  fact  a  high  level  of  recognition.   The  capacity  of  humans 
to  recognize  previously  viewed  pictures  has  been  well  established  (Shepard,  1967; 
Standing,  Conezio,  &  Haber,  1970).   Although  the  PSSs  used  were  not  intended  to 
duplicate  currently  used  icons,  the  similarity  between  the  study  materials  and 
commercially  available  software  icons  indicates  that  this  is  a  plausible  explanation. 
Another  possibility  is  that  the  basic  metaphors  used  for  the  word  processing 
commands  represented  as  analogical  PSSs  were  particularly  appropriate  to  the  domain 
represented.    This  could  account  for  the  high  level  of  interpretation  among  subjects 
who  were  not  categorized  as  having  graphic  interface  experience.    Although  the  icons 
may  not  have  been  recognized  from  previous  use,  the  pictorial  sign  symbol  may  have 


90 
represented  the  manual  equivalent  of  the  computer  function  so  well  that  the  subjects 

familiar  with  the  domain  of  editing  text  could  correctly  make  the  association  with  the 

word  processing  command.   These  explanations  take  into  account  both  the  higher 

overall  interpretation  scores  for  analogical  PSSs  as  well  as  the  higher  overall  scores 

for  subjects  with  graphic  interface  experience. 

A  possible  explanation  for  the  lack  of  improvement  in  interpretation  of 
analogical  PSSs  over  time  may  be  the  lack  of  feedback  during  the  experiment.    A 
simpler  explanation  is  that  the  research  design,  which  was  intended  to  negate  practice 
effects  and  differential  carryover  effects,  was  successful  in  doing  so.   There  was  no 
practice  effect  observed  for  analogical  PSSs,  and  no  significant  differences  between 
the  average  of  correct  interpretations  at  different  sequence  positions. 

The  interpretation  of  logical  PSSs  followed  a  different  pattern.    An  increase  in 
correct  interpretations  between  the  first  and  second  positions  was  observed,  but  there 
was  no  change  in  correct  interpretations  between  the  second  and  third  sequence 
positions.   This  suggests  the  possibility  that  the  additional  exposure  to  the  same 
concepts  in  different  types  of  pictures  aided  in  decoding  and  interpretation,  but  the 
lack  of  feedback  and  the  arbitrary  nature  of  the  representations  used  in  logical  sign 
symbols  prevented  additional  improvement  on  the  third  viewing  sequence  position. 
Additional  studies  could  be  designed  to  confirm  or  disconfirm  this  possibility  and  to 
further  explore  the  processes  by  which  people  seek  to  gain  meaning  from  the  human- 
computer  interface. 
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How  can  one  reconcile  the  different  patterns  of  interpretation  for  realistic, 

analogical,  and  logical  PSSs?  The  scores  for  the  analogical  PSSs  can  be  accounted 

for  by  the  research  design,  which  included  the  cycling  of  symbols  to  negate  practice 

effects  and  differential  carryover  effects.    Analogies  and  metaphors  used  in  this  type 

of  PSS  are  highly  productive  in  creating  insight  (Cassidy  &  Knowlton,  1983),  which 

may  account  for  the  high  level  of  correct  interpretation. 

The  scores  for  interpretation  of  realistic  and  logical  PSSs  may  be  explained  by 
the  recognition  of  patterns  by  the  subjects.    A  range  of  related  pictures,  in  context, 
may  serve  as  exemplars,  thus  facilitating  concept  acquisition  (Cassidy  &  Knowlton, 
1983).    Because  the  same  12  concepts  were  presented  at  each  sequence  position,  and 
the  realistic  PSSs  represented  the  concepts  in  a  consistent  manner,  it  seems  plausible 
that  pattern  recognition  or  concept  attainment  could  have  occurred  for  this  type  of 
symbol.   With  each  exposure  to  the  same  concepts,  the  subjects  had  additional 
instances  in  which  to  make  a  decision  regarding  the  intended  concepts  (Bruner, 
Goodnow,  &  Austin,  1956).    Logical  PSSs,  using  arbitrary  representation  of  visual 
elements,  would  have  to  be  taught,  much  as  visual  language  literacy  is  taught. 

In  summary,  each  of  the  different  functional  PSS  types  revealed  a  different 
pattern  of  correct  interpretations  over  the  three  viewing  sequence  positions.    Realistic 
PSSs  were  interpreted  more  accurately  at  each  subsequent  viewing.    Analogical  PSSs 
were  consistently  interpreted  most  accurately  among  the  three  types,  but  showed  no 
increase  in  correct  interpretations  over  time.    Logical  PSSs  were  the  least  successfully 
interpreted  symbols,  and  showed  no  increase  in  correct  interpretations  after  the  second 
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group  viewed.    Subjects  with  graphics  interface  experience  were  better  able  to 

interpret  PSSs  than  subjects  with  text-only  experience  or  subjects  classified  as 

novices. 

Implications 

These  results  add  to  the  knowledge  base  used  by  professionals  involved  in  the 
design  and  development  of  computer-based  instructional  materials.    As  computer 
systems  are  utilized  by  teachers  and  students,  the  importance  of  interface  designs  that 
are  easily  understood  and  easy  to  learn  will  increase.    Results  of  this  study  and  some 
implications  of  these  results  are  discussed  below.    In  considering  these  implications 
and  recommendations,  one  should  keep  in  mind  the  limitations  to  generalization 
represented  by  the  population  sampled,  the  content  domain  of  word  processing,  and 
the  specific  PSSs  created  for  this  study.    Given  these  caveats,  the  following 
conclusions  seem  tenable. 

The  results  of  this  study  support  the  use  of  Knowlton's  taxonomy  of  the 
functional  types  of  pictures  as  a  viable  scheme  for  the  systematic  design  and 
evaluation  of  pictorial  sign  symbols  of  the  types  used  to  convey  messages  by  graphic 
elements  in  human-computer  interfaces.   This  implies  that  practitioners  should  review 
and  reconsider  guidelines  for  interface  design  in  terms  of  this  taxonomy.    Specifically, 
guidelines  referring  to  the  continuum  of  concreteness  to  abstraction  need  to  be 
considered  within  the  functional  types  of  realistic  and  analogical  pictorial  sign 
symbols. 
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The  results  of  this  study  indicate  that  experience  with  computer  systems  with 

graphic  user  interfaces  affects  the  ability  to  interpret  PSSs  used  within  the  interface 

regardless  of  the  functional  type  of  picture.   This  implies  that  designers  need  to  be 

aware  of  the  background  experience  and  characteristics  of  the  intended  audience  when 

designing  pictures  for  use  as  PSSs,  if  the  intended  users  are  to  be  able  to  interpret  the 

meanings  of  the  interface  elements  regardless  of  the  presence  of  text  labels. 

The  results  of  this  study  indicate  that  analogical  PSSs  representing  word 
processing  and  text  editing  commands  were  interpreted  more  accurately  than  either 
realistic  or  logical  PSSs  by  teacher  education  students.    This  implies  that  PSSs  that 
refer  to  computer  processes  by  analogy  to  closely  related  manual  processes  may  be 
more  effectively  interpreted  than  other  functional  types  of  pictures.    This  suggests  that 
designers  should,  when  possible,  create  symbols  that  closely  represent  the  manual 
actions  that  the  computer  program  function  is  intended  to  supplant. 

Within  the  experimental  time  frame,  the  number  of  subsequent  correct 
interpretations  of  realistic  PSSs  increased  significantly  between  adjacent  sequence 
positions.    This  suggests  that  designers  should  create  symbols  that  represent  the  actual 
appearance  of  the  parts  of  the  computer  system  to  which  the  symbols  refer,  when 
reference  to  a  familiar  manual  analogy  is  not  possible. 

Within  the  experimental  conditions,  logical  PSSs  were  the  least  well 
interpreted  and  were  not  consistently  interpreted  more  accurately  on  subsequent 
viewing  sequence  positions.   This  implies  that  designers  should  avoid  the  use  of 
arbitrary  representation  of  major  elements  for  commands  within  the  human-computer 
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interface.   This  does  not  imply  that  logical  representations  of  the  patterns  of 

arrangement  or  orders  of  connection  of  elements  within  a  PSS  should  be  avoided. 

Recommendations  for  Future  Research 

The  primary  purpose  of  this  study  was  to  determine  whether  Knowlton's 
taxonomy  of  pictures  by  functional  types  could  be  used  to  analyze  the  PSSs  used  in 
human-computer  interfaces,  because  previous  schema  had  resulted  in  apparently 
contradictory  results.    Much  of  the  prior  research  in  this  broad  area  was  difficult  to 
interpret  and  generalize  due  to  the  nonequivalence  of  the  constructs  used  and  the 
confounding  of  theoretical  constructs  with  specific  tasks  and  applications.   The 
positive  results  of  this  investigation  suggest  that  systematic  investigation  of  specific 
questions  involving  carefully  defined  populations,  specific  content  and  task  domains, 
and  systematically  constructed  PSSs  can  produce  meaningful  results.    Therefore,  the 
following  questions  are  raised  as  possible  directions  for  future  research. 

At  what  point  does  the  sequence  effect  reach  a  peak  or  plateau?  Would 
repetition  of  the  same  PSSs  affect  the  ability  to  interpret  on  subsequent  viewings?   In 
this  study,  realistic  PSSs  were  interpreted  more  successfully  at  each  subsequent 
measurement,  but  never  surpassed  the  interpretation  level  of  analogical  PSSs.   Over 
time,  would  realistic  PSSs  be  more  effectively  interpreted  than  analogical  PSSs? 

To  what  extent  are  the  results  of  this  study  generalizable  beyond  the  population 
studied?   Can  the  results  be  replicated  across  linguistic  and  cultural  groups,  across  age 
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and  educational  levels?   Could  elementary  or  secondary  school  students  interpret  PSSs 

for  word  processing  commands  as  well  as  university  students? 

How  does  the  consistency  of  presentation  and  relationship  of  PSSs  within  a  set 
affect  interpretation?  Given  complete  sets  of  symbols  instead  of  randomized  groups, 
would  the  differences  between  functional  types  of  PSSs  remain  or  be  negated? 

Would  the  provision  of  informative  or  corrective  feedback  affect  the  ability  of 
individuals  with  differing  types  of  experience  to  interpret  PSSs  over  subsequent 
exposures?   Are  there  other  factors  that  might  result  in  further  interactions?   Would 
the  measurement  of  computer  literacy  as  a  covariate  provide  additional  information? 

Does  the  viewing  context  affect  the  ability  to  interpret  PSSs  in  a  more 
naturalistic  setting?   For  instance,  would  the  implementation  of  a  graphic  interface 
without  text  labels  provide  a  more  meaningful  context  than  the  written  instructions 
used  in  this  study?   Would  a  more  meaningful  and  realistic  context  result  in  a 
significant  effect  on  the  ability  to  interpret  PSSs? 

How  would  prior  experience  affect  interpretation  if  subjects  were  trained  to 
competency  in  the  use  of  a  text  and  menu-based  word  processing  application  and  then 
provided  with  an  equivalent  system  lacking  text  labels  but  containing  PSSs  as 
equivalent  commands?   This  suggests  operationalizing  competency  as  a  covariate. 

How  would  the  content  domain  affect  interpretation?   This  study  used  word 
processing  software  as  a  content  domain.    Would  similar  results  be  found  within 
database,  spreadsheet,  statistical  analysis,  presentation  graphics,  or  other  software 
packages  for  different  content  and  task  areas? 
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How  would  additional  attributes  of  PSSs,  such  as  color  coding,  textural  and 

contrast  cues,  animation,  or  status  cues  affect  the  interpretation  of  these  symbols? 

Would  the  display  of  a  single  PSS  indicating  the  current  status  of  an  element,  or  the 

display  of  all  possible  states,  with  the  current  status  indicated  by  highlighting  or 

boldness,  affect  interpretation? 

How  would  the  implementation  of  an  "intelligent"  interface,  in  which  only 

those  options  that  were  currently  available,  affect  the  interpretation  of  those  options  if 

they  were  represented  by  PSSs?   For  instance,  if  a  person  highlighted  a  section  of  text 

on  a  word  processing  screen,  the  PSSs  for  "Cut,"  "Copy,"  "Font,"  and  "Style"  might 

automatically  appear  near  the  highlighted  material,  so  the  person  using  the  software 

could  make  an  appropriate  menu  choice  with  little  additional  movement  or  effort. 

Would  the  functional  type  of  PSS  used  in  this  situation  have  an  effect  upon  correct 

interpretation  and  error-free  selection  of  the  desired  command? 

Summary 

This  study  investigated  the  relationship  of  the  type  of  pictorial  sign  symbol 
classified  by  Knowlton's  (1964,  1966)  taxonomy  and  the  type  of  computer  experience 
to  the  ability  to  interpret  pictures  representing  word  processing  software  commands, 
under  differing  context  conditions.    Teacher  education  students  enrolled  in  an 
introduction  to  educational  media  course  were  chosen  as  a  population  of  interest.    The 
statistical  analysis  used  a  four-way  ANOVA  based  ona3x2x3x3  factorial  design 
with  repeated  measures.    Factors  measured  included  computer  experience  type  (three 
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levels),  context  (two  levels),  pictorial  sign  symbol  type  (three  levels),  and  sequence  of 

viewing  (three  levels). 

Main  effects  were  observed  for  computer  experience  type,  pictorial  sign 
symbol  type,  and  sequence.    A  significant  interaction  was  observed  between  the 
sequence  and  pictorial  sign  symbol  variables.   There  was  no  effect  due  to  context  as 
operationalized  for  this  study. 

Interpretation  of  the  results  of  this  study  were  made  with  acknowledgement  to 
the  limitations  imposed  by  the  population  studied  and  the  content  domain  represented 
by  the  PSSs.   Within  those  limitations,  analogical  PSSs  were  more  effectively 
interpreted  by  subjects  with  all  types  of  previous  experience,  and  logical  PSSs  were 
the  least  well  interpreted  by  all  subjects.    Realistic  PSSs  were  interpreted  more 
accurately  during  the  second  and  third  sequence  of  viewing  positions  than  during  the 
initial  presentation.    Prior  experience  with  a  graphic  user  interface  positively  affected 
the  ability  to  interpret  PSSs. 

Conclusions  from  this  study  were  restricted  to  the  variables  as  operationalized 
and  manipulated  in  this  study,  and  limited  to  the  population  represented  by  the  sample 
participating  in  the  study.    Further  research  questions  intended  to  clarify  questions 
raised  by  the  results  of  this  study  as  well  as  explore  additional  constructs,  contexts, 
and  content  domains  were  posed  as  directions  for  future  research. 

In  conclusion,  the  conduct  of  this  study  generated  evidence  that  may  be  useful 
to  those  engaged  in  designing  visual  messages  within  human-computer  interfaces  for 
teacher  education  students.   The  evidence  gathered  indicates  that  the  materials  and 


98 
instruments  developed  for  this  study  are  reliable  as  administered.    The  results  of  this 

study  provide  support  for  the  systematic  use  of  Knowlton's  taxonomy  in  the  design 

and  evaluation  of  pictorial  sign  symbols,  and  the  limitations  of  this  study  indicate 

appropriate  directions  for  further  research. 
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